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Abstract 

The  Air  Force  is  pursuing  several  efficiency  initiatives  designed  to  reduce  support 
function  costs.  One  such  initiative  is  an  effort  to  reduce  the  flow  days  of  items  being 
repaired  in  the  Air  Force’s  organic  depots.  Many  end  items  are  affected  by  awaiting  parts 
(AWP)  delays,  which  increase  total  flow  days.  The  first  step  in  reducing  AWP  delays  is 
to  identify  which  piece  parts  are  causing  the  delays. 

A  gap  analysis  was  conducted  to  identify  a  process  for  creating  a  list  of  piece 
parts  that  are  causing  AWP  delays.  In  addition,  a  clinimetric  method  was  used  to  develop 
an  aggregate  measure  of  AWP  impact  by  which  the  list  of  piece  parts  could  be 
prioritized.  The  gap  analysis  showed  that  such  a  list  can  be  created  with  Cognos,  a 
reporting  tool  currently  used  by  the  depots,  which  can  pull  data  from  multiple 
information  systems.  In  addition,  only  minor  changes  to  infonnation  recorded 
throughout  the  repair  process  are  needed.  An  aggregate  measure  of  AWP  impact  was 
also  created  and  tested.  It  produced  significantly  different  prioritizations  from  the 
individual  constituent  variables,  and  provides  a  possible  method  for  helping  depot 
managers  to  understand  decision  tradeoffs  between  different  parts  shortage  priorities. 
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IDENTIFYING,  TRACKING,  AND  PRIORITIZING  PARTS  UNAVAILABILITY 


I.  Introduction 


1.1  Background 

The  Air  Force’s  organic  depots  suffer  from  increased  flow  days  due  to  awaiting 
parts  (AWP).  In  fiscal  year  2012,  the  76th  Commodities  Maintenance  Group  experienced 
more  than  505,000  AWP  days  across  479  end  items.  Reducing  AWP  time  on  end  items 
being  repaired  would  reduce  the  total  shop  flow  days  and  inventory  costs.  The  Air  Force 
recently  undertook  several  efficiency  initiatives  to  reduce  overhead  and  support  function 
costs  (Department  of  Defense,  2011).  One  of  those  initiatives  was  to  reduce  costs  in 
Logistics  and  Infrastructure,  specifically  through  Supply  Chain  Management  (SCM) 
efficiencies.  This  includes  a  concerted  effort  to  reduce  the  flow  days  of  items  being 
repaired  in  the  Air  Force’s  organic  depots  (Frickson,  2012).  Many  process  improvement 
efforts  have  yielded  outstanding  initial  results,  and  AWP  time  reduction  is  an  extension 
of  this  effort. 

Reducing  flow  days  is  an  effective  means  of  reducing  costs  through  inventory 
reductions.  This  concept  is  based  on  Little’s  Law,  which  shows  that  the  amount  of 
inventory  in  a  depot  is  a  product  of  flow  time  and  flow  rate  (Little,  1961 :  383). 

Therefore,  if  the  flow  time  (shop  flow  days  in  this  case)  is  reduced,  inventory  in  the  depot 
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will  be  reduced.  In  addition  to  cost  savings,  reducing  flow  days  increases  the 
responsiveness  of  the  depots  by  repairing  end  items  faster.  The  problem  of  AWP  time  is 
not  new.  It  represents  a  classic  tradeoff  between  inventory  cost  and  responsiveness  to  the 
customer  (the  warfighter).  Massive  amounts  of  inventory  can  be  held  to  ensure  that  parts 
are  available  for  end  item  repairs  when  and  where  needed,  which  ensures  the  highest 
possible  responsiveness  to  warfighter  needs.  However,  this  is  not  a  practical  solution  to 
the  AWP  problem.  Such  inventory  stores  would  cause  support  costs  to  soar  and  it  would 
not  be  sustainable,  especially  given  the  current  fiscal  situation.  As  a  result,  the  Air  Force 
must  find  an  appropriate  balance  between  inventory  cost  and  responsiveness  to  the 
warfighter. 

1.2  Definitions 

For  the  purposes  of  this  study,  “AWP”  is  meant  in  the  broadest  sense.  It  includes 
the  fonnal  AWP  process  used  by  the  Air  Force  and  the  Defense  Logistics  Agency  (DLA), 
and  it  includes  any  situation  where  an  end  item  is  not  able  to  be  repaired  due  to  a  part(s) 
not  being  physically  available  to  the  mechanic.  Any  exceptions  to  this  definition  will  be 
explicitly  stated. 

1.3  Problem  Statement 

To  attempt  to  strike  the  balance  between  inventory  costs  and  warfighter  needs,  the 
Air  Force  uses  multiple  information  systems  (ESS,  EXPRESS,  ABOM,  NIMMS,  ITS, 
D035,  D200,  etc.)  and  reams  of  instructions,  policies,  and  procedures  (AFMAN  23-110, 
EXPRESS  Information  Handbook,  MAJCOM  OIs,  ALC  OIs,  Local  OIs,  etc.).  The 
combination  of  the  two  creates  a  fairly  automated  supply  chain.  However,  despite  the 
quality  of  the  information  systems,  instructions,  policies,  and  procedures,  there  are  still 
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numerous  cases  of  parts  not  being  available  when  and  where  they  are  needed.  The  76th 
Commodities  Maintenance  Group  currently  has  615  piece  parts  backordered  against  362 
end  items.  Much  effort  has  been  and  is  still  expended  on  correcting  these  errors  and 
creating  a  better  supply  chain.  Regardless  of  how  successful  these  initiatives  have  been 
or  will  be  there  will  always  be  incidences  where  the  supply  chain  fails  to  provide  the  right 
parts,  in  the  right  place,  in  the  right  quantity,  and  at  the  right  time.  In  short,  there  will 
always  be  AWP  delays. 

The  depots  can  identify  which  end  items  are  consistently  put  into  the  fonnal  AWP 
process  and  how  long  they  spend  in  the  process.  However,  they  cannot  identify  which 
parts  consistently  cause  end  items  to  enter  the  formal  AWP  process.  In  addition,  they 
cannot  identify  which  parts  consistently  cause  AWP  delays  aside  from  the  fonnal  AWP 
process.  The  current  information  systems  do  not  track  AWP  delays  that  are  projected  to 
be  less  than  10  days.  Finally,  the  depots  do  not  have  a  way  to  prioritize  which  parts  cause 
the  most  problems.  Therefore,  they  need  a  process  to  identify  which  parts  shortages  have 
the  greatest  negative  impact  on  their  operations.  Such  a  capability  would  allow  the 
depots  to  conduct  a  Pareto  analysis  to  focus  problem  solving  efforts  on  the  worst 
problems. 

This  research  develops  a  process  for  creating  a  prioritized  list  of  parts  that  have 
the  greatest  negative  impact  on  their  operations.  The  process  will  allow  depots  to  create  a 
list  of  which  parts  are  causing  AWP  delays,  which  is  the  first  step  towards  reducing  AWP 
delays  in  the  depots.  In  addition,  this  research  will  develop  a  measure  (metric)  of  AWP 
impact.  This  metric  can  then  be  used  to  prioritize  the  list  of  parts  by  their  impact.  This 
can  allow  the  depots  to  identify  to  their  suppliers  which  parts  are  most  important  for  them 
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to  acquire  for  immediate  use,  thus  increasing  responsiveness  to  the  warfighter.  It  will 
also  allow  them  to  monitor  the  list  over  time,  so  trends  may  be  identified,  inventory 
investments  may  be  justified,  or  problem  solving  efforts  can  be  focused.  This  has  the 
added  benefit  of  providing  a  feedback  loop  to  Air  Force  demand  planners/forecasters. 

1.4  Research  Scope 

The  scope  of  this  research  will  be  limited  to  end  items  repaired  in  the  76th 
Commodities  Maintenance  Group  at  the  Oklahoma  City  Air  Logistics  Complex.  This 
limitation  is  necessary  given  the  amount  of  variation  in  systems,  processes,  parts,  and 
procedures  across  the  depots  and  even  within  the  Oklahoma  City  Air  Logistics  Complex. 
However,  this  research  is  intended  to  provide  a  template  for  other  units  within  the 
Oklahoma  City  Air  Logistics  Complex  and  the  other  two  Air  Logistics  Complexes. 

1.5  Approach 

This  research  consists  of  two  main  parts:  a  gap  analysis  of  the  infonnation 
available  and  the  information  needed  to  create  a  list  of  piece  parts  causing  AWP  delays; 
and  the  development  of  a  measure  for  prioritizing  the  parts  list.  First,  it  is  necessary  to 
detennine  what  information  is  needed  to  develop  a  list  of  parts  causing  AWP  delays. 
Then,  the  information  that  is  available  in  the  current  infonnation  systems  must  be 
identified  through  process/information  mapping  of  the  end  items  in  the  repair  process. 
Once  the  current  state  and  the  desired  end  state  are  identified  as  described  above,  a  gap 
analysis  between  the  two  states  will  be  conducted.  If  no  gaps  exist,  then  a  process  for 
retrieving  and  organizing  the  necessary  data  will  be  created.  If  there  are  infonnation 
gaps,  then  recommendations  on  how  to  close  those  gaps  will  be  developed.  Finally,  a 
measurement  of  AWP  impact  will  be  developed.  AWP  impact  cannot  be  defined  except 
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by  the  instrument  used  to  measure  it.  Therefore,  this  measure  will  be  developed  using  a 
pragmatic  approach.  This  will  involve  combining  several  observable  attributes  (manifest 
variables)  in  the  proper  fashion  to  detennine  the  value  of  the  unobservable  attribute 
(latent  variable),  which  will  be  called  the  AWP  Impact  Index  (Hand,  2004:  15,  52,  152). 

1.6  Thesis  Outline 

This  thesis  consists  of  five  chapters:  an  introduction,  a  literature  review,  an 
explanation  of  the  research  methodology,  an  explanation  of  the  analysis  and  research 
results,  and  conclusions  based  on  the  results.  Chapter  two  contains  a  review  of  literature 
pertaining  to  the  various  supply  management  models  the  Air  Force  uses,  efforts  to 
address  the  Air  Force’s  supply  chain  shortcomings,  forecasting  principles,  and 
measurement  theory.  Chapter  three  describes  how  the  gap  analysis  was  conducted,  how 
information  was  gathered,  and  how  the  AWP  Impact  Index  was  developed.  Chapter  four 
explains  what  information  was  gathered  and  presents  the  results  of  the  gap  analysis.  It 
also  provides  a  detailed  explanation  of  the  development  of  the  AWP  Impact  Index. 
Chapter  five  will  recommend  a  process  to  close  any  information  gaps  identified  in 
chapter  four  and  describe  the  process  by  which  a  prioritized  parts  list  can  be  created. 
Finally,  it  will  recommend  several  methods  for  prioritizing  the  parts  list. 
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II. 


Literature  Review 


2.1  Overview 

In  identifying  and  analyzing  the  problem,  a  review  of  the  literature  and  Air  Force 
programs  regarding  awaiting  parts  and  the  overall  supply  process  was  conducted.  This 
included  literature  on  the  various  models  and  systems  the  Air  Force  uses  to  manage  its 
supply  chain.  In  addition,  literature  pertaining  to  forecasting  principles  and  Air  Force 
demand  planning,  particularly  feedback  loops,  was  reviewed  to  ensure  the  process 
developed  could  be  used  as  a  feedback  loop  for  Air  Force  demand  planners,  if  desired. 
Finally,  a  literature  review  of  measurement  theory  was  conducted  to  ensure  the  AWP 
Impact  Index  development  was  consistent  with  existing  theory  and  practices. 

2.2  Awaiting  Parts  Research/Programs 

Much  research  has  been  done  to  provide  a  better  model  or  system  to  avoid  or 

minimize  AWP  delays.  Regardless  of  the  quality  of  a  supply  chain  management  model 

or  system,  there  will  necessarily  always  be  some  AWP  delays  because  of  the  tradeoff 

between  inventory  costs  and  warfighter  needs.  The  question  is  how  many  AWP  delays 

are  acceptable  and  what  should  be  done  to  address  those  delays?  An  article  in  the 

Journal  of  Air  Force  Logistics  entitled  “Why  So  Many  AWP  LRUs?”  discusses  how  the 

Air  Force  seems  to  consistently  have  too  many  line-replaceable  units  (LRUs)  and  shop- 

replaceable  units  (SRUs)  in  formal  AWP  status.  The  authors  provide  a  very  good 

summary  of  how  end  items  are  chosen  for  repair  in  the  depots: 

The  Execution  and  Prioritization  of  Repair  Support  System  (EXPRESS)  was 
implemented  throughout  AFMC  depots  starting  in  1995.  Its  logic  prioritizes 
warfighter  needs  for  LRUs  to  be  supplied  from  the  depot  component  repair 
program.  By  netting  out  all  LRU  repair  pipelines,  the  system  determines  each  day 
what  each  repair  shop  should  induct.  After  it  determines  the  prioritized  induction 
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list  of  LRUs  for  each  shop,  it  proceeds  down  the  prioritized  induction  list  of 
LRUs  to  see  if  the  needed  resources  are  in  place  to  execute  the  repair  of  each  item 
on  the  list.  If  one  of  the  resources  is  unavailable,  the  system  skips  over  that  LRU 
and  checks  the  next  item  on  the  list.  This  process  continues  down  the  prioritized 
list  to  find  any  LRU  that  has  all  the  depot  resources  to  do  the  repair.  That 
successful  list  is  sent  to  the  D035  Express  Table  for  immediate  induction  into  the 
repair  shop.  (Carter  and  London,  2003:3 1) 

The  authors  further  explain  why  this  process  is  flawed.  EXPRESS  relies  on 
probabilities  to  detennine  which  parts  are  needed.  As  a  result,  an  end  item  could  be 
prevented  from  being  repaired  due  to  the  unavailability  of  a  part  even  though  that  part 
may  not  be  needed.  The  parts  actually  needed  are  determined  when  it  enters  the  repair 
process.  In  addition,  they  explain  that  stock  levels  are  based  on  historical  demand,  but 
many  parts  are  not  stocked  because  they  do  not  have  any  or  much  historical  demand. 

This  makes  sense  if  historical  demand  is  a  good  predictor  of  future  demand.  The  authors 
argue  “historical  usage  is  not  a  good  indicator  of  the  real  requirement”  (Carter  and 
London,  2003:  3 1).  In  addition,  even  if  the  historical  demand  was  a  good  indicator  of 
future  demand,  the  use  of  probabilities  to  detennine  the  parts  required  for  repair 
inherently  includes  a  certain  amount  of  enor.  This  highlights  the  fact  that  there  will  be 
AWP  delays  and  the  depots  must  be  able  to  respond  to  those  delays  in  a  timely  and 
effective  manner. 

An  additional  benefit  this  article  provides  is  an  explanation  of  the  delays  created 
by  EXPRESS’S  induction  logic.  An  end  item  that  is  first  on  the  prioritized  list  of  LRUs 
to  be  repaired  could  be  inducted  behind  an  end  item  with  a  lower  priority  if  all  the  parts 
were  not  available  for  the  higher  priority  end  item.  This  leads  to  higher  priority  items 
being  passed  over  for  lower  priority  items  just  because  all  the  parts  it  predicts  it  will  need 
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are  not  available.  As  a  result,  some  end  items  do  not  enter  the  formal  AWP  system  as 
soon  as  they  should,  but  are  in  fact  delayed  entering  the  depot  repair  process  due  to  parts 
not  being  available.  If  the  end  item  does  not  enter  fonnal  AWP  status,  then  it  does  not 
receive  the  extra  attention  it  deserves.  While  this  does  not  create  a  delay  in  the  depot,  it 
does  delay  that  end  item  before  it  even  enters  the  depot  repair  process.  This  delay  should 
be  recorded  and  tracked  along  with  the  delays  caused  once  an  end  item  enters  the  repair 
process. 

A  current  effort  to  capture  the  effect  of  AWP  time  and  to  counter  it  is  called  Deep 
Look.  It  was  started  by  Defense  Logistics  Agency  at  the  Ogden  Air  Logistics  Complex, 
refined,  and  passed  on  to  the  Oklahoma  City  Air  Logistics  Complex  and  the  Warner 
Robins  Air  Logistics  Complex.  It  was  started  in  2009  and  the  Oklahoma  City  Air 
Logistics  Complex  Deep  Look  Center  was  stood  up  in  January  2012  (Schwing,  2012). 
Deep  Look  addresses  the  same  problem  as  this  thesis,  but  from  the  end  item  perspective. 
The  program  develops  a  prioritized  list  of  end  items  based  on  warfighter  needs.  The 
members  of  the  Deep  Look  Center  then  do  a  deep  look  using  a  holistic  approach  to 
detennine  where  the  problem  lies,  such  as  “materials,  manpower,  equipment,  facilities, 
technical  data,  etc.”  (Kinkade  et  al,  2009:  3). 

Deep  Look’s  intent  is  not  to  address  the  AWP  problem,  specifically.  For  each  end 
item,  there  might  be  several  parts  that  require  attention.  There  is  no  distinction  made 
between  the  parts  that  are  needed  though.  In  other  words,  each  part  is  treated  as  equally 
important.  From  an  end  item  perspective,  this  is  perfectly  logical.  However,  the  equal 
treatment  of  each  part  ignores  the  effect  the  parts  have  on  other  end  items.  The  effect  on 
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multiple  end  items  of  one  particular  part  not  being  available  is  precisely  what  this  thesis 
attempts  to  capture. 

2.3  Air  Force  Supply  Chain  Models/Systems 

The  Air  Force  uses  several  models  and  systems  to  attempt  to  strike  the  balance 
between  inventory  costs  and  warfighter  needs.  Each  of  these  models  and  systems  try  to 
prevent  the  need  for  an  AWP  process.  Blazer  and  Sloan  provide  an  overview  of  four 
models  the  Air  Force  uses  to  provide  the  right  parts  in  the  right  place  at  the  right  time  for 
the  right  cost.  The  Aircraft  Availability  Model  (AAM),  Aircraft  Sustainment  Model 
(ASM),  Equipment  Prioritization,  and  Customer-Oriented  Leveling  Technique  (COLT) 
are  the  four  models  discussed.  Given  a  set  amount  of  funding,  the  Aircraft  Availability 
Model  uses  marginal  analysis  to  detennine  which  end  item  should  be  purchased  next. 

The  next  best  item  to  purchase  is  based  on  how  much  that  item  will  increase  aircraft 
availability  per  dollar  spent,  or  based  on  the  most  “bang  per  buck”  (Blazer  and  Sloan, 
2007:  68).  Its  purpose  is  to  ensure  that  the  limited  amount  of  funding  available  is  spent  in 
an  optimum  manner.  This  should  provide  the  Air  Force  the  maximum  aircraft 
availability  given  the  budget  constraints.  However,  this  model  inherently  concedes  that 
AWP  delays  will  still  occur.  The  Aircraft  Sustainability  Model  is  very  similar  to  Aircraft 
Availability  Model,  but  its  purpose  is  limited  to  planning  wartime  spares  requirements. 
Again,  this  model  aims  to  prevent  AWP  delays,  but  it  inherently  accepts  that  some  will 
occur.  The  Equipment  Prioritization  Model  uses  marginal  analysis  to  detennine  which 
equipment  items  to  buy  given  the  funding  constraints.  While  the  model  does  not  involve 
buying  parts  for  aircraft,  it  does  show  another  example  of  the  Air  Force  attempting  to 
minimize  shortages,  but  accepting  that  shortages  will  still  occur.  Finally,  the  Customer- 
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Oriented  Leveling  Technique  detennines  stock  levels  at  depots  and  bases  with  the 
objective  of  minimizing  customer  wait  time.  It  operates  under  the  same  budget 
constraints  as  the  previous  models,  but  it  aims  to  optimize  the  quantity  and  mix  of  parts  at 
each  location  (bases  and  depots).  All  four  of  these  models  attempt  to  minimize  AWP 
delays  and  they  all  accept  that  AWP  delays  will  still  occur  because  resources  are  limited. 
However,  none  of  them  address  what  to  do  when  those  delays  occur  or  how  to  measure 
the  delays  because  those  are  not  their  purposes  (Blazer  and  Sloan,  2007:  68-70). 

A  report  by  the  Logistics  Management  Institute  further  discusses  leveling 
techniques  using  lean  logistics  principles.  Instead  of  a  total  reengineering  of  Air  Force 
systems  and  procedures,  the  report  advocates  the  careful  implementation  of  readiness- 
based  leveling  (RBL)  in  order  to  complement  the  D04 1  system,  which  calculates  the  total 
worldwide  spares  requirement.  Interestingly,  the  report  explicitly  discusses  the  shortages 
that  are  expected  with  the  Air  Force’s  various  supply  chain  models.  “Lean  Logistics 
focuses  more  on  quick  response  to  parts  shortages  when  they  occur  rather  than  on 
attempting  to  preclude  shortages  by  investment  in  large  inventories”  (O’Malley,  1996:  2). 
This  statement  highlights  the  choice  between  inventory  and  responsiveness,  and  it  argues 
that  variability  (shortages)  should  be  countered  by  responsiveness  whenever  possible. 
While  the  report  does  not  advocate  removing  all  Air  Force  legacy  systems,  it  does  state 
that  “their  lack  of  timeliness  is  an  impediment  to  realizing  the  benefits  of  Lean  Logistics 
and  must  be  overcome”  (O’Malley,  1996:  2). 

For  example,  the  Air  Force’s  current  systems  and  procedures  capture  and  track  a 
parts  shortage  at  the  depot  if  it  enters  the  fonnal  AWP  process.  However,  this  is  only 
implemented  if  an  item  is  estimated  to  take  10  days  or  more  to  acquire  (76th  Maintenance 
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Wing,  2012).  This  creates  the  possibility  of  hundreds,  if  not  thousands  of  days  in  delays 
without  ever  being  identified  and  tracked.  The  10-day  standard  for  entry  into  the  formal 
AWP  process  is  intended  to  limit  the  end  items  in  fonnal  AWP  status  to  only  those  that 
are  expected  to  experience  severe  delays.  In  this  case,  the  Defense  Logistics  Agency  has 
defined  “severe”  as  10  days  or  more  (76th  Maintenance  Wing,  2012).  However,  this 
standard  ignores  the  aggregate  impact  of  small  AWP  delays  and  it  hides  the  small  delays 
in  daily  activities,  making  them  difficult  to  address.  In  addition,  some  units  do  not  have  a 
formal  AWP  process,  such  as  the  76th  Propulsion  Maintenance  Group.  Also,  the  fonnal 
AWP  process  is  inconsistently  applied,  such  as  waiting  30  days  to  enter  an  end  item  into 
formal  AWP  status  (see  section  4.3).  These  examples  show  that  some  AWP  delays  are 
not  immediately  identified,  if  at  all,  or  actively  tracked.  If  a  delay  is  not  identified,  the 
supply  chain  cannot  respond  to  it.  This  is  not  to  say,  the  Air  Force  does  not  respond  to 
parts  shortages  at  all.  However,  it  does  not  adequately  identify,  track,  prioritize,  and 
respond  to  all  parts  shortages.  According  to  Womack  and  Jones  in  their  book  Lean 
Thinking,  “transparency  in  everything  is  a  key  principle”  (Womack  and  Jones,  2003:  97). 
In  other  words,  lean  principles  dictate  that  any  delay  should  be  given  immediate  attention 
and  be  aggressively  attacked.  Currently,  the  Air  Force  lacks  the  full  awareness  and  level 
of  detail  necessary  to  even  have  the  opportunity  to  respond  to  such  delays.  This  research 
aims  to  provide  the  Air  Force  with  such  opportunities. 

2.4  Forecasting  Principles  and  Air  Force  Demand  Planning 

A  brief  discussion  of  forecasting  principles  is  appropriate  for  two  reasons.  First, 
as  demonstrated  in  the  previous  section,  the  Air  Force  has  invested  much  time  and  money 
to  develop  a  robust  supply  chain,  to  include  a  forecasting  process.  Second,  one  of  the 
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intents  of  this  research  is  to  develop  a  feedback  loop  that  might  be  useful  to  forecasters. 
Business  Forecasting,  written  by  J.  Holton  Wilson  and  Barry  Keating,  discusses  a  9  step 
forecasting  process.  Their  ninth  step  is  “Tracking  Results.”  They  state  that  “Deviations 
from  the  forecast  and  the  actual  events  should  be  discussed  in  an  open,  objective,  and 
positive  manner”  (Wilson  and  Keating,  2009:  490).  This  is  consistent  with  lean 
principles  as  well.  The  authors  go  on  to  explain  that  these  discussions  need  to  be  for  the 
purpose  of  improving  future  forecasts.  An  understanding  of  why  the  actual  data  deviates 
from  the  forecasts  is  necessary.  In  addition,  one  must  detennine  if  the  size  of  the 
deviation  is  significant  enough  to  warrant  changes  to  the  model  or  the  forecasting  process 
(Wilson  and  Keating,  2009:  490-491).  The  point  being  made  by  the  authors  is  that 
feedback  on  the  quality  and  accuracy  of  the  forecast  is  not  only  useful,  but  essential  to 
detennining  the  validity  of  the  forecasting  model/process. 

The  source  of  feedback  also  matters.  “It  is  important  to  stress. .  .the  critical  role 
that  communication  and  cooperation  between  managers  and  technicians  play  in  building 
and  maintaining  a  successful  forecasting  process”  (Wilson  and  Keating,  2009:  491).  The 
commitment  to  communication  is  also  critical  because  a  forecast  model’s  validity  can  and 
most  likely  will  change  over  time.  Again,  the  quality  of  the  Air  Force’s  forecasting 
models  is  not  in  question,  but  proper  validation  of  them  through  continuous  feedback  is 
absolutely  necessary.  The  formal  AWP  process  does  not  provide  full  feedback  to  the  Air 
Force’s  demand  planners.  In  addition,  some  units  do  not  have  a  fonnal  AWP  process, 
such  as  the  76th  Propulsion  Maintenance  Group.  As  a  result,  the  full  “magnitude  of  the 
errors”  cannot  be  determined,  which  does  not  allow  for  a  proper  assessment  of  the 
forecasting  model  (Wilson  and  Keating,  2009:  490). 
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The  primary  forecasting  model  used  by  the  Air  Force  is  the  Reparability  Forecast 
Model  (RFM).  The  RFM  operates  similar  to  a  Material  Requirements  Planning  (MRP) 
system,  where  it  takes  inputs  from  various  systems  that  tell  it  what  parts  are  needed, 
when  they  are  needed,  how  long  it  takes  to  get  them,  and  if  the  parts  are  in  stock.  The 
output  is  a  report  showing  what  the  consumable  shortfalls  are,  as  depicted  in  Figure  1 
(Gaudette  et  al,  2002:  7). 


Figure  1.  Inputs  and  Outputs  of  RFM  with  MRP  Equivalents  in  Parentheses  (Gaudette  et 

al,  2002:  7) 


The  Reparability  Forecast  Model  does  not  detennine  stock  levels  or  conduct  any 
ordering.  It  simply  forecasts  the  need  based  on  projected  repairs  and  determines  any 
shortfalls  based  on  actual  stock  levels  and  due-ins  (Gaudette  et  al,  2002:  5-7).  Similar  to 
EXPRESS,  the  Reparability  Forecast  Model  uses  probabilities  and  averages  to  determine 
what  parts  are  needed.  This  is  done  out  of  necessity.  Material  Requirements  Planning 
(MRP)  systems  are  best  suited  for  manufacturing  environments  where  the  Bill  of 
Materials  (BOM)  provides  a  definitive  list  of  what  parts  will  be  needed.  Since  the 
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Reparability  Forecast  Model  tries  to  predict  repair  parts  needed,  there  will  almost 
certainly  be  differences  between  what  it  projects  is  needed  and  what  is  actually  needed. 
The  authors  accept  this  error  in  the  system  because  the  Law  of  Large  Numbers  works  in 
the  favor  of  the  Reparability  Forecast  Model  (Gaudette  et  al,  2002:  7).  Due  to  the  sheer 
volume  of  work  at  the  depots,  using  probabilities  and  averages  is  fairly  accurate  over 
time.  However,  there  is  still  variability  in  the  population,  which  is  why  a  feedback  loop 
is  necessary. 

Figure  1  shows  that  no  direct  feedback  loop  exists  for  the  Reparability  Forecast 
Model.  “Materiel  managers  can  generate  special  requisitions  and  expedite  existing 
requisitions  to  meet  consumable  demands  for  repairs”  (Gaudette  et  al,  2002:  8).  This 
means  that  shortages  are  identified  to  the  individuals  that  control  the  available  quantities, 
so  the  shortages  do  not  have  the  opportunity  to  cause  delays.  However,  this  information 
is  not  directly  fed  back  to  the  infonnation  systems  that  produce  the  forecasts,  which  is 
depicted  in  Figure  1 .  It  is  possible  to  add  a  feedback  loop  to  the  process.  The 
Reparability  Forecast  Model  only  identifies  parts  shortages;  it  does  not  order  any  parts  to 
compensate  for  the  projected  shortages.  Item  managers,  on  the  other  hand,  can  and  do 
place  special  orders  if  they  deem  it  necessary  to  compensate  for  the  projected  shortages. 
If  the  item  managers  were  provided  with  a  list  of  parts  that  were  currently  short  or  that 
have  been  short  over  a  period  of  time,  just  as  this  thesis  suggests,  it  could  serve  as  a 
feedback  loop.  In  other  words,  if  item  managers  were  able  to  identify  how  many  times 
and  for  how  long  their  items  caused  work  stoppages  despite  the  Reparability  Forecast 
ModeFs  predictions  and  their  special  orders,  item  managers  would  be  able  to  consider 
that  in  future  orders  for  those  parts. 
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There  has  been  some  recognition  of  the  Air  Force’s  need  for  a  feedback  loop. 
Kaczmarek  et  al  argue  that  the  Air  Force  needs  to  conduct  customer-demand-oriented 
logistics  planning.  They  lay  out  a  five  step  methodology  for  demand  planning.  The  fifth 
step  is  a  feedback  loop.  The  authors  explain  that  a  feedback  loop  “is  key  to  any  process 
for  the  purposes  of  monitoring  progress  and  making  process  improvements. 

Additionally,  feedback  highlights  the  essential  nature  of  demand  planning  as  a 
continuously  iterative  process  of  planning,  execution,  assessment,  and  improvement  in 
forecasting  and  meeting  demand”  (Kaczmarek  et  al,  2002:  20).  The  authors  implemented 
their  method  for  the  F101  engine  at  the  Oklahoma  City  Air  Logistics  Complex.  The 
process  advocated  in  the  article  produced  reduced  flow  days,  decreased  response  times, 
and  had  no  production  stoppages  when  given  30  days’  notice  of  a  requirement 
(Kaczmarek  et  al,  2002:  20).  This  shows  that  when  provided  a  feedback  loop,  significant 
operational  improvements  can  be  made. 

Kaczmarek  et  al  also  noted  two  challenges  to  implementation:  Infonnation 
Technology  and  Change  Management.  These  challenges  provide  a  warning  for  the 
progression  of  this  research.  According  to  the  authors,  extensive  manual  work  was 
needed  to  get  the  infonnation  systems  to  do  what  was  required  of  them.  Finding  a  simple 
way  to  get  the  data  needed  from  the  infonnation  systems  is  critical  to  making  the 
feedback  loop  useful  for  “rapid  decision  making  capability”  (Kaczmarek  et  al,  2002:  20). 
Also,  fonnal  training  was  recommended  for  what  infonnation  to  put  in  the  systems  and 
why  it  is  important  (Kaczmarek  et  al,  2002:  20). 
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2.5  Measurement  Theory 

Assuming  it  is  possible  to  develop  a  list  of  parts  that  cause  work  stoppages,  the 
next  task  is  to  prioritize  that  list.  There  are  many  ways  to  prioritize  a  list,  and  the 
correctness  of  the  prioritization  method  is  relative  to  what  one  is  trying  to  measure. 
Measurement  theory  is  defined  in  Introduction  to  Measurement  Theory  as  “a  branch  of 
applied  statistics  that  attempts  to  describe,  categorize,  and  evaluate  the  quality  of 
measurements,  improve  the  usefulness,  accuracy,  and  meaningfulness  of  measurements, 
and  propose  methods  for  developing  new  and  better  measurement  instruments”  (Allen 
and  Yen,  1979:  2).  One  part  of  this  thesis  attempts  to  develop  a  process  that  allows  for 
the  collection  of  data  that  can  be  measured.  Without  the  data,  measurement  theory  is 
useless.  Dr.  David  Hand  touches  on  this  in  his  book,  Measurement  Theory  and  Practice: 
the  World  through  Quantification,  by  stating  that  “measurement  is  the  activity  which 
produces  the  raw  material  which  statistical  methods  analyze”  (Hand,  2004:  19).  This 
thesis  aims  to  provide  the  raw  material  necessary  for  statistical  analysis,  which  may  guide 
managers’  decisions  by  improving  their  understanding  of  what  is  occurring.  This  task 
must  be  done  carefully  and  methodically  because  measurements  can  also  be  misleading 
and  lessen  a  manager’s  understanding  of  what  is  occurring.  Dr.  Hand  explains  this 
dilemma.  “The  measurement  process  extracts  just  one  aspect  of  the  object  it  is  measuring 
and  assigns  a  number  to  that  aspect.  But  things  in  the  real  world  are  not  described  by  a 
single  aspect”  (Hand,  2004:  12).  This  is  a  major  limitation  of  any  measurement  (or 
metric).  If  not  selected  carefully,  a  measurement  may  actually  do  harm.  In  addition  to 
selecting  a  measurement  carefully,  one  must  explain  and  understand  the  purpose  of  a 
measurement. 
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According  to  Dr.  Hand,  there  are  two  measurement  approaches:  representational 
and  pragmatic.  “The  representational  aspect  refers  to  the  extent  to  which  the  numbers  are 
chosen  so  that  the  relationships  between  them  reflect  the  relationships  between  the 
objects  being  studied,  in  terms  of  the  attribute  in  question”  (Hand,  2004:  25).  For 
example,  the  number  of  times  various  parts  cause  work  stoppages  could  be  compared  to 
detennine  which  part  has  the  biggest  negative  impact.  “In  pragmatic  measurement 
models  the  measurement  procedure  entirely  defines  the  variable  being  measured”  (Hand, 
2004:  52).  For  example,  I  develop  a  way  to  measure  impact  of  AWP  delays  on  the  depot 
repair  process. 

The  method  for  developing  a  measure  is  dependent  upon  numerous  factors,  which 
include  the  objective  of  the  measure  and  whether  the  measure  must  be  direct  or  indirect 
(Hand,  2004:  84).  Therefore,  the  first  step  is  to  define  the  objective  of  the  measure.  In 
this  case,  the  objective  is  to  develop  a  measurement  of  AWP  impact  which  can  be  used  to 
prioritize  a  list  of  piece  parts  causing  AWP  delays.  Next,  it  must  be  detennined  if  this 
can  be  measured  directly  or  indirectly.  There  are  many  attributes  that  can  be  directly 
measured  with  regard  to  AWP  impact,  such  as  the  amount  of  AWP  time  a  part  caused, 
the  number  of  times  it  caused  an  end  item  to  be  AWP  (frequency),  the  customer  priority 
of  the  end  item  affected,  the  quantity  needed,  etc.  However,  it  would  be  difficult  to  argue 
that  only  one  of  these  direct  measures  is  the  best  measurement  by  which  to  prioritize  the 
list  of  piece  parts.  According  to  Hand,  “in  many  situations,  however,  direct  measurement 
of  the  attribute  in  question  is  not  possible,  and  indirect  strategies,  in  which  its  value  is 
deduced  from  its  relationship  to  other  attributes,  which  can  be  directly  measured,  must  be 
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used”  (2004:  87).  As  a  result,  several  direct  measures  will  be  combined  to  develop  an 
indirect  measure  of  AWP  impact. 

Since  AWP  impact  is  defined  by  how  it  is  measured,  a  pragmatic  approach  is 
appropriate  (Hand,  2004:  52).  More  specifically,  the  clinimetric  approach  appears  best 
suited  for  the  problem  in  this  research  (Hand,  2004:  15;  Fayers  and  Hand,  2002:  240- 
241).  Clinimetric  methods  are  defined  by  Fayers  and  Hand  as  attempting  “to  summarize 
multiple  attributes  with  a  single  index”  (2002:  241).  That  is  precisely  what  this  research 
attempts  to  do.  Much  of  the  literature  regarding  measurement  theory,  particularly  with 
regard  to  clinimetrics,  is  based  on  work  in  the  behavioral  sciences  and  medicine,  but  it 
can  be  easily  adapted  and  applied  to  other  fields  as  well  (Hand,  2004;  Fayers  and  Hand, 
2002;  Allen  and  Yen,  1979;  Wright  and  Feinstein,  1992). 

Fayers  and  Hand  explain  that  the  creators  of  such  a  measurement  should  choose 
the  variables  to  be  included  and  they  should  choose  the  “relative  importance”  of  the 
variables  (2002:  241).  This  should  all  be  based  on  the  objective  of  the  measurement 
(2002:  241).  In  addition,  they  argue  that  this  seemingly  arbitrary  selection  method  is  not 
a  weakness  of  the  method  because  it  forces  the  “constructors  of  a  scale  to  decide  exactly 
what  it  is  they  want  their  index  to  measure,  and  to  make  this  public”  (2002:  241).  Wright 
and  Feinstein  are  even  more  explicit  about  how  such  indexes  should  be  developed.  They 
explain  that  the  clinimetric  method  for  developing  multiple-item  indexes  is  generally 
done  in  three  stages.  “The  first  step  considers  a  pool  of  candidate  variables;  the  second 
step  chooses  the  final  items  retained  for  the  index;  and  the  third  step  determines  the 
relative  emphasis,  or  ‘weight’,  given  to  the  retained  items”  (Wright  and  Feinstein,  1992: 
1205).  They  recommend  several  methods  for  completing  each  step,  such  as  using 
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individual  judgment  or  a  panel  of  experts  (1992:  1205-1206).  Section  3.2  explains  how 
the  three-stage  approach  was  applied  to  this  research. 
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III.  Methodology 


3.1  Overview 

This  chapter  will  describe  the  methodology  used  for  this  study.  It  describes  the 
AWP  Impact  Index  development,  validation,  and  testing.  Then,  it  describes  the  gap 
analysis,  which  determined  the  differences  between  the  current  state  and  the  desired  end 
state.  This  analysis  identified  the  information  gaps  that  currently  prevent  the  depots  from 
tracking  AWP  delays  by  piece  part. 

3.2  AWP  Impact  Index  Development 

In  keeping  with  the  three-stage  approach  outlined  in  section  2.5, 1  selected  the 
pool  of  candidate  variables  based  on  knowledge  and  intuition  gained  during  the  gap 
analysis.  As  the  value  of  the  candidate  variable  increases,  so  does  a  parts’  priority  on  the 
parts  list.  For  example,  I  included  AWP  time  in  such  a  way  that  as  the  amount  of  AWP 
time  caused  by  a  piece  part  increases,  so  does  its  priority  on  the  parts  list  relative  to  the 
other  parts.  Next,  the  76th  Commodities  Maintenance  Group  Deputy  Commander 
selected  subject  matter  experts  to  serve  as  the  expert  panel.  The  76th  CMXG  Deputy 
Commander  selected  the  panel  members  based  on  their  knowledge  and  experience 
regarding  AWP  delays.  The  panel  members  included  the  76th  CMXG  Deputy 
Commander,  a  76th  CMXG  Senior  Management  Analyst,  and  a  76th  CMXG 
Management  Analyst.  The  panel  of  experts  selected  the  final  variables,  defined  the 
measurement  methods,  and  detennined  the  relative  importance  of  each  variable.  The 
researcher  assigned  weights  to  each  variable  based  on  the  relative  importance  assigned  by 
the  expert  panel.  As  the  importance  decreased,  so  did  the  weights.  However,  depot 
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personnel  must  determine  what  weights  are  appropriate  prior  to  use.  The  weights  should 
be  adjusted  until  depot  personnel  deem  the  resulting  prioritizations  correct. 

3.3  AWP  Impact  Index  Significance  Test 

The  final  variables  selected  were  readily  available  direct  measurements.  The 
parts  list  could  be  prioritized  based  on  these  measurements  individually,  but  that  would 
ignore  the  other  variables.  In  other  words,  the  list  could  be  prioritized  by  AWP  time,  but 
that  would  ignore  the  customer  priority  and  vice  versa.  Therefore,  it  must  be  shown  that 
the  AWP  Impact  Index  produces  a  significantly  different  prioritization  than  the  individual 
variables.  This  requires  the  use  of  rank  correlation  methods.  I  used  the  two-tailed 
hypothesis  test  recommended  for  Speannan’s  Rank  Correlation  Coefficient  method  to 
test  if  the  differences  among  rankings  based  on  the  AWP  Impact  Index  and  the  individual 
variables  were  statistically  significant.  Three  sources  were  consulted  for  this  method. 

The  tenninology  and  variables  from  McClave  et  al  is  used  in  this  research,  but  the 
application  of  the  method  is  consistent  with  all  three  sources  (Kendall  and  Gibbons, 

1990;  McClave  et  al,  2011;  Kutner  et  al,  2005). 

Speannan’s  Rank  Correlation  Coefficient  uses  various  sums  of  squares  between 
ranks  in  order  to  estimate  the  linear  correlation  coefficient  between  two  sets  of  rankings. 
The  conelation  coefficient  can  range  between  -1  (perfect  negative  conelation)  and  +1 
(perfect  positive  correlation).  The  closer  the  coefficient  is  to  -1  or  +1,  the  more 
conelated  two  rankings  are.  The  closer  the  coefficient  is  to  zero,  the  less  correlated  two 
rankings  are  (McClave  et  al,  2011:  14 — 37-40).  Finally,  there  are  two  conditions  that 
must  be  met  in  order  to  properly  test  the  significance  of  the  Speannan’s  rank  conelation 
coefficients.  First,  the  sample  must  be  randomly  selected.  Second,  the  probability 
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distributions  of  the  two  variables  selected  must  be  continuous  (McClave  et  al,  2011:  14 — 
42). 

In  this  case,  piece  parts  must  be  randomly  selected  from  the  data.  There  is  no 
specific  discussion  of  recommended  sample  sizes  in  the  literature,  but  the  sample  sizes  of 
the  examples  provided  in  the  literature  ranged  from  10  to  20  (McClave  et  al,  2011:  14 — 
37-44;  Kutner  et  al,  2005:  87-89;  Kendall  and  Gibbons,  1990:  69-71).  Therefore,  20 
piece  parts  were  randomly  selected  to  ensure  the  sample  size  was  large  enough.  The  data 
provided  by  the  76th  Commodities  Maintenance  Group  included  615  piece  parts.  The 
researcher  listed  the  parts  in  ascending  order  in  Microsoft  Excel  by  the  National  Stock 
Number  (NSN),  which  is  an  alphanumeric  assigned  to  each  piece  part  in  the  Department 
of  Defense.  The  RAND()  function  in  Excel  assigned  a  random  number  between  0  and  1 
to  each  piece  part.  Next,  I  sorted  by  the  random  number  in  ascending  order  and  selected 
the  first  20  piece  parts  to  use  as  the  random  sample  from  the  population. 

The  next  step  was  to  detennine  the  correlation  coefficient  for  each  pairing  of  the 
variables.  There  are  two  equations  that  can  be  used  to  calculate  the  correlation 
coefficient.  One  is  the  “full”  equation;  the  other  is  a  shortcut  equation  that  is  appropriate 
when  there  are  no  ties  in. 


Where 


SSi , 


V  ssuussv 


rs  =  sample  rank  correlation  coefficient 
p  =  population  rank  correlation  coefficient 
SSUV  =  -  u)(vt  -  v) 
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ssuu  =  2f®iC Ui  -u)2 

SSVV  =  Zf=nC Vf  -  R)2 

Ui  =  Rank  of  /th  observation  in  sample  1 
v,  =  Rank  of  z'th  observation  in  sample  2 
The  method  requires  assigning  the  average  of  the  applicable  ranks  to  any  piece 
parts  that  have  tied  measurements.  For  example,  if  the  quantity  for  three  piece  parts  is  5, 
and  they  occupy  the  rankings  5,  6,  and  7,  each  part  is  assigned  the  rank  of  6.  Then,  the 
researcher  used  a  two-tailed  hypothesis  test  to  detennine  each  coefficient’s  statistical 
significance  (a  =  0.005).  With  a  two-tailed  hypothesis  test  and  an  alpha  of  0.005,  the 
critical  value  is  0.612  (Zar,  1972:  578-80).  The  test  is  outlined  below: 

H0:  p  =  0 
Ha:  p  *  0 

Test  Statistic:  rs,  the  sample  rank  correlation  coefficient 
Rejection  Region:  \rs\  >  rs000S/2  =  0.612 

This  hypothesis  test  involves  a  pair  wise  comparison  of  each  possible  combination  of  the 
methods  used  to  rank  the  piece  parts.  In  section  4.2,  the  expert  panel  selected  five 
variables  for  calculating  the  AWP  Impact  Index.  Therefore,  fifteen  |C(6,2)}  pair  wise 
comparisons  must  be  made.  The  piece  part  rankings  provided  by  each  variable  and  the 
AWP  Impact  Index  must  be  compared  with  each  other  to  detennine  if  the  AWP  Impact 
Index  produces  rankings  that  are  significantly  different.  In  addition,  the  comparisons 
identify  if  there  are  any  variables’  piece  part  rankings  that  are  not  significantly  different. 
As  a  result  of  making  fifteen  simultaneous  comparisons,  when  the  null  hypothesis  is 
rejected,  the  confidence  of  the  rejection  must  be  adjusted  to  reflect  the  number  of 
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rejections.  The  Bonferroni  method  shows  that  the  statement  of  family  confidence  for  the 
rejections  made  is  1  -  ga,  where  g  is  the  number  of  rejections  (Kutner  et  al,  2005:  155- 
157).  For  example,  Table  1  shows  that  7  correlation  coefficients  were  significantly 
correlated.  The  level  of  confidence  of  this  statement  is  1  -  (7)(0.005)  or  0.965. 

3.4  AWP  Impact  Index  Practical  Comparisons 

In  addition  to  the  statistical  rank  correlation  method  used  above,  the  researcher 
conducted  three  practical  comparisons.  The  Spearman’s  rank  correlation  coefficient  is 
useful  for  determining  if  the  differences  in  rankings  are  statistically  significant,  when 
comparing  the  same  randomly  selected  parts.  However,  it  does  not  allow  for  a 
comparison  of  how  all  piece  parts  are  ranked  across  all  variables.  I  could  not  find  a 
statistical  test  for  such  a  comparison,  so  I  created  three  practical  comparisons.  In  these 
practical  comparisons,  I  ranked  the  entire  population  of  piece  parts  by  the  AWP  Impact 
Index,  which  is  developed  in  section  4.2,  and  each  variable  individually. 

The  first  comparison  method  used  the  same  20  piece  parts  as  the  Spearman’s 
method.  The  researcher  compared  the  rankings  of  those  20  piece  parts  across  all 
variables  to  the  rankings  assigned  by  the  AWP  Impact  Index,  since  it  is  the  measurement 
under  consideration.  I  then  calculated  the  difference  between  the  rank  assigned  by  AWP 
Impact  Index  and  the  individual  variables.  This  allowed  for  a  positive  or  negative  result. 
Intuitively,  if  the  AWP  Impact  Index  ranked  the  parts  properly,  the  absolute  value  of  the 
difference  should  be  larger  as  the  relative  importance  of  the  variable  decreases.  This  aids 
in  determining  if  the  AWP  Impact  Index  applied  the  relative  importance  of  each  variable 
properly.  This  also  allows  for  a  comparison  of  how  each  variable  ranked  the  same  part  in 
relation  to  the  entire  population  of  data. 
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The  second  comparison  identified  commonalities,  instead  of  differences.  Again, 
the  researcher  compared  the  AWP  Impact  Index  rankings  and  the  rankings  by  the  five 
variables  using  the  entire  population.  This  involved  comparing  the  top  20  piece  parts  for 
each  variable  to  the  top  20  piece  parts  as  ranked  by  the  AWP  Impact  Index.  I  searched 
for  the  top  20  piece  parts  identified  by  the  AWP  Impact  Index  in  each  of  the  other 
variables’  top  20  piece  parts.  I  counted  and  highlighted  any  instance  of  a  piece  part 
commonality  in  the  top  20  by  each  variable. 

The  logic  behind  this  method  is  that  the  depot  leadership  will  mostly  be 
concerned  with  only  the  most  troublesome  parts,  but  they  will  also  look  at  more  than  just 
the  highest  ranked  piece  part.  Therefore,  it’s  useful  to  understand  how  much 
commonality  there  is  among  the  top  rankings.  This  is  particularly  useful  as  priorities 
change.  For  example,  if  warfighter  priority  becomes  more  important  than  cost,  then 
depot  leadership  should  want  to  know  if  the  rankings  by  Spares  Priority  Release 
Sequence  priority  are  the  same  as  the  ranking  by  cost  ratio.  Otherwise,  switching 
prioritization  methods  may  not  give  different  priorities. 

The  third  comparison  identified  agreement  by  quintile.  The  author  ranked  each  of 
the  607  piece  parts  according  to  the  AWP  Impact  Index  and  each  variable.  The  rankings 
were  then  divided  into  5  quintiles.  I  compared  the  rankings  by  each  variable  to  the 
ranking  by  the  AWP  Impact  Index.  I  calculated  the  number  of  times  the  same  piece  part 
was  in  the  same  quintile  for  each  respective  comparison.  This  showed  the  amount  of 
agreement  by  quintile  for  each  comparison.  This  method  provided  a  better  understanding 
of  the  agreement  across  the  entire  population  of  data  as  opposed  to  20  randomly  selected 
piece  parts  or  the  top  20  piece  parts. 
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However,  it’s  important  to  remember  that  these  three  comparisons  only  highlight 
the  differences  or  commonalities  among  the  rankings.  They  do  not  identify  which 
measurement  is  better  or  best.  The  accuracy  of  the  rankings  assigned  to  each  part  can 
only  be  determined  by  the  users  of  the  measurements,  since  there  is  no  universal  or 
completely  objective  standard  for  measuring  AWP  impact. 

3.5  Gap  Analysis 

The  gap  analysis  identified  the  differences  between  the  desired  end  state  and  the 
current  state  of  the  information  recorded  throughout  the  repair  process.  Specifically,  the 
gap  analysis  identified  where  work  stoppages  (delays)  can  occur  in  the  repair  process  due 
to  parts  unavailability  and  what  information  is  recorded  about  those  delays.  The 
researcher  used  this  information  to  describe  the  current  state.  Next,  I  defined  the  desired 
end  state.  Finally,  I  compared  the  two  states  to  determine  what  differences  existed. 

Prior  to  conducting  the  gap  analysis,  an  understanding  had  to  be  gained  of  the 
basic  repair  process  that  end  items  follow  once  they  are  inducted  into  the  76th 
Commodities  Maintenance  Group  (CMXG).  The  leadership  of  the  76th  CMXG 
identified  managers  and  supervisors  within  the  76th  CMXG  as  subject  matter  experts 
(SMEs).  These  SMEs  explained  the  basic  repair  process.  The  SMEs  created  a  flow  chart 
of  the  process  (depicted  in  Figure  2). 


Figure  2.  Flow  Chart  of  Repair  Process 
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With  the  basic  flow  chart  established,  it  was  necessary  to  detennine  where  AWP 
delays  could  occur.  The  researcher  created  a  tentative  flow  chart  with  delay  locations 
through  emails  and  phone  calls  with  various  SMEs  within  the  552nd  Commodities 
Maintenance  Squadron  (CMMXS),  which  is  a  unit  in  the  76th  Commodities  Maintenance 
Group.  However,  I  modified  the  flow  chart  and  delay  locations  after  I  conducted  on-site 
discussions  with  schedulers.  The  researcher  held  discussions  with  schedulers  from  the 
fuels  accessories  shop,  governors  and  accessories  shop,  and  constant  speed  drives  (CSD) 
shop.  The  subject  matter  experts  selected  these  shops  because  their  processes  presented 
the  widest  range  of  variability  to  the  basic  process  in  Figure  2.  In  other  words,  studying 
the  processes  within  these  three  shops  gave  the  best  understanding  of  the  rest  of  the  shops 
in  the  76th  Commodities  Maintenance  Group.  The  researcher  asked  schedulers  to 
describe  any  process  variations  that  could  occur.  In  some  cases,  we  brought  work  center 
supervisors  into  the  discussion  to  clear  up  any  uncertain  infonnation.  Throughout  all 
these  discussions,  I  specifically  asked  for  statements  of  fact  only  and  strictly  avoided 
individual  opinions  and  conjecture  to  ensure  I  gained  an  unbiased  understanding  of  the 
process.  Then,  I  verified  the  locations  of  possible  AWP  delays.  This  step  was  critical  to 
developing  the  desired  end  state  and  to  focus  the  infonnation  mapping  to  only  the 
pertinent  steps  in  the  repair  process. 

Next,  the  author  examined  the  same  three  shops  in  detail  to  determine  the  current 
state.  I  examined  each  step  in  the  repair  process  depicted  in  Figure  2,  except  Test  and 
Sell.  I  did  not  examine  the  Test  and  Sell  steps  because  no  AWP  delays  can  occur  in 
them.  Subject  matter  experts,  such  as  schedulers,  work  leaders,  and  mechanics  explained 
what  occurred  at  each  step  of  the  repair  process.  They  also  explained  any  possible 
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deviations  from  the  basic  repair  process  in  Figure  2.  This  ensured  that  any 
recommendations  provided  in  Chapter  5  would  account  for  the  variation  in  process 
execution.  After  I  visited  all  three  shops,  I  created  process  flow  charts  for  each  shop  and 
then  a  consolidated  flow  chart  depicting  what  information  is  recorded  at  each  step  and  the 
infonnation  system  in  which  it  is  recorded. 

The  same  subject  matter  experts  verified  the  accuracy  of  the  flow  charts.  In 
addition,  the  applicable  infonnation  systems  offices  of  primary  responsibility  (OPRs)  at 
Tinker  AFB  verified  the  accuracy  of  the  flow  charts.  The  system  OPRs  validated  that  the 
infonnation  was  actually  recorded  where  the  subject  matter  experts  stated  it  was 
recorded.  The  researcher  discussed  the  accuracy  of  the  data  with  the  subject  matter 
experts  and  system  OPRs  as  well,  so  the  validity  and  reliability  of  the  information 
recorded  could  be  ascertained. 

Then,  I  defined  the  desired  end  state.  It  must  be  reiterated  that  the  objective  of 
this  gap  analysis  was  not  to  change  the  repair  process.  The  objective  of  this  analysis  was 
to  detennine  what  infonnation  should  be  recorded  and  in  what  information  system(s)  it 
should  be  recorded,  so  a  list  of  piece  parts  causing  delays  could  be  created.  The  variables 
identified  by  the  panel  of  experts  discussed  in  section  3.2  drove  what  information  should 
be  recorded  about  the  delays.  The  desired  end  state  had  to  include  all  the  information 
necessary  to  use  the  AWP  Impact  Index. 

Finally,  the  researcher  compared  the  two  states  to  identify  the  gaps  that  existed 
between  them.  With  the  gaps  identified,  the  final  step  was  to  develop  a  process  to  close 
the  gaps.  I  developed  two  possible  solutions.  The  same  subject  matter  experts  and 
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information  systems  offices  of  primary  responsibility  validated  the  solutions  for 
feasibility,  validity,  and  cost  (time  and  money). 
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IV.  Analysis  and  Results 


4.1  Overview 

This  chapter  provides  a  detailed  explanation  of  the  development  of  the  AWP 
Impact  Index,  the  significance  of  the  index  prioritizations,  the  comparisons  with  the 
individual  variables,  and  the  gap  analysis  results.  The  expert  panel  determined  that  the 
AWP  Impact  Index  should  include  AWP  time,  quantity,  frequency,  warfighter  priority, 
and  cost  ratio.  The  testing  results  show  that  the  AWP  Impact  Index  produced  priorities 
significantly  different  from  the  individual  variables,  except  AWP  time.  In  addition,  the 
gap  analysis  identified  where  AWP  delays  can  occur,  the  current  state,  and  desired  end 
state  of  infonnation  being  recorded  regarding  AWP  delays.  The  results  show  that  the  gap 
between  the  infonnation  currently  being  recorded  about  AWP  delays  and  the  desired  end 
state  is  not  that  wide.  In  the  governors  and  accessories  shop,  all  information  needed  to 
track  AWP  delays  by  piece  part  is  already  being  recorded.  In  the  fuels  accessories  and 
CSD  shops,  the  kitting  step  is  the  only  step  that  needs  more  information  recorded  to 
achieve  the  desired  end  state. 

4.2  AWP  Impact  Index  Development 

The  researcher  developed  the  AWP  Impact  Index  in  three  stages.  First,  I 
compiled  a  list  of  variables  based  on  knowledge  and  intuition  gained  during  the  gap 
analysis.  Second,  the  76th  Commodities  Maintenance  Group  assembled  an  expert  panel 
to  select  the  final  variables  and  how  to  measure  each  of  them.  Third,  the  panel  of  experts 
determined  the  relative  importance  of  each  variable.  Finally,  the  AWP  Impact  Index  was 
tested  to  detennine  if  it  produced  results  that  were  significantly  different  than  the 
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individual  variables  and  to  see  if  the  variables  were  significantly  different  from  each 
other. 

4.2.1  Initial  Variable  Selection 

Any  variable  selected  that  indicates  increased  AWP  impact  should  cause  the 
part’s  index  value  to  increase.  As  a  result,  the  researcher  selected  the  variables  below  for 
consideration  by  the  panel  of  experts: 

•  Increased  Cost  Ratio  (Ce/Cp) 

•  End  Item  Cost  (CE) 

•  Piece  Part  Cost  (Cp) 

•  Increased  Quantity  Needed  (Q) 

•  Increased  AWP  Time  (T) 

•  Increased  SPRS  Priority  (S) 

•  Increased  Frequency  (FP) 

The  cost  ratio  is  meant  to  represent  the  tradeoff  between  buying  more  end  items  or  more 
piece  parts  to  ensure  the  customers’  needs  are  met.  For  example,  it  is  better  to  buy  more 
of  a  piece  part  that  costs  ten  cents  than  to  buy  more  of  an  end  item  that  costs  $  10,000.  In 
addition,  it  would  be  foolish  to  have  a  ten  cent  part  delay  the  repair  of  a  $10,000  end 
item.  Therefore,  as  the  cost  ratio  increases,  so  should  the  piece  part’s  priority.  The 
quantity  of  a  piece  part  needed  should  also  drive  up  its  priority  because  as  the  quantity 
increases,  so  does  the  severity  and  impact  of  the  shortage.  The  AWP  time  is  an  important 
factor  to  include  in  the  index  because  as  the  AWP  time  increases,  so  does  the  severity  and 
impact  of  the  shortage.  The  frequency  also  contributes  to  the  severity  and  impact  of  the 
shortage  because  frequent,  short  delays  can  have  the  same  impact  as  one  long  delay. 
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Finally,  the  Air  Force  assigns  a  Spares  Priority  Release  Sequence  (SPRS)  priority  to  very 
high  priority  end  items.  EXPRESS  creates  a  prioritized  list  of  end  items  that  need 
repaired  based  on  the  factors  discussed  in  section  2.2,  but  the  Spares  Priority  Release 
Sequence  “reorders  item  priorities  based  on  specific  field  requisitions.  The  priority 
sequence  assigns  a  priority  to  each  item  needed  in  the  total  worldwide  requirement”  (Air 
Force  Materiel  Command,  2006:  4).  The  Spares  Priority  Release  Sequence  priority 
values  range  from  0  to  84.  As  the  value  increases,  so  does  the  priority  of  the  end  item 
(Air  Force  Materiel  Command,  2006:  25-26).  Therefore,  the  Spares  Priority  Release 
Sequence  priority  is  a  direct  measure  of  customer  priorities  and  as  it  increases,  so  should 
the  priority  of  the  piece  part. 

4.2.2  Final  Variable  Selection  by  Expert  Panel 

The  researcher  presented  the  variables  listed  in  section  4.2.1  to  the  panel  of 
experts  to  assess  if  the  variables  were  appropriate  to  be  included  in  the  index  and  if  any 
variables  needed  to  be  added.  The  panel  detennined  that  all  five  variables  presented 
should  be  included  in  the  AWP  Impact  Index  and  that  no  other  variables  should  be  added. 
The  panel  also  defined  how  each  variable  should  be  measured.  They  determined  that  the 
cost  ratio  should  be  measured  with  the  sell  price  of  the  end  item  and  the  purchase  price  of 
the  piece  part.  The  panel  decided  that  the  quantity  variable  should  count  the  piece  parts 
shortage  quantity  when  attributed  to  an  AWP  delay.  They  also  detennined  that  AWP 
time  should  be  measured  from  the  start  of  any  delay  to  the  repair  process  due  to  parts 
unavailability  (i.e.-when  an  R06  delay  code  is  entered  in  the  Time,  Attendance,  and 
Accountability  system)  and  end  when  the  last  part  needed  to  recommence  repairs  arrived. 
In  addition,  the  panel  determined  that  the  AWP  time  should  only  be  recorded  against  the 
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last  piece  part  to  arrive.  The  panel  decided  that  the  frequency  should  count  the  number  of 
times  a  piece  part  caused  a  delay  (i.e.  -  an  end  item  gets  delayed  three  times  by  three 
different  parts;  one  delay  would  be  recorded  against  each  part). 

4.2.3  Relative  Importance  of  Variables 

The  panel  also  detennined  the  relative  importance  of  each  variable,  which  are 
listed  below  in  descending  order  or  importance: 

1 .  Increased  Frequency  (FP) 

2.  Increased  AWP  Time  (T) 

3.  Increased  Quantity  Needed  (Q) 

4.  Increased  Cost  Ratio  (Ce/Cp) 

5.  Increased  SPRS  Priority  (S) 

4.2.4  Data  Collection  for  Variables 

The  optimal  method  to  compare  the  various  prioritizations  created  by  the 
individual,  direct  measurements  would  be  to  use  actual  data.  However,  since  the  primary 
focus  of  this  research  is  to  identify  if  such  data  can  even  be  obtained,  actual  data  was  not 
available  for  all  variables.  The  76th  Commodities  Maintenance  Group  provided  data  on 
the  current  end  items  in  formal  AWP  status.  The  data  is  current  as  of  January  27,  2013 
and  included  the  end  item  National  Stock  Numbers,  piece  part  National  Stock  Numbers, 
formal  AWP  time  (days),  quantity  of  piece  parts  needed,  and  the  frequency  of  AWP 
delays.  The  76th  Commodities  Maintenance  Group  also  provided  the  end  item  sell 
prices,  which  they  pulled  on  January  3 1,  2013.  In  addition,  Air  Force  Materiel  Command 
provided  the  piece  part  purchase  prices  and  the  Spares  Priority  Release  Sequence 
priorities  of  the  end  items  from  EXPRESS,  which  is  accurate  as  of  February  4,  2013. 
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Actual  AWP  time  data  was  used,  but  it  was  not  measured  in  the  same  manner  as  it 
is  described  in  section  4.2.2.  The  AWP  time  data  only  measured  the  number  of  days  in 
formal  AWP  status.  However,  this  is  an  understatement  of  the  total  AWP  time.  The 
AWP  time  data  used  ranged  from  18  days  to  999  days.  This  prevents  the  evaluation  of 
the  impact  of  many  small  delays.  In  addition,  the  system  used  by  the  Oklahoma  City  Air 
Logistics  Complex  to  track  fonnal  AWP  days  stops  counting  time  after  999  days.  This 
prevents  the  evaluation  of  the  impact  of  delays  beyond  999  days.  As  a  result,  it  is 
unknown  what  impact,  if  any,  the  extreme  values  for  AWP  time  will  have  on  the 
prioritization  of  parts.  However,  this  weakness  is  somewhat  mitigated  by  the  fact  that  the 
AWP  time  data  used  contains  a  wide  range  of  values. 

In  addition,  the  research  could  not  always  attribute  the  AWP  delay  to  just  one  part 
since  the  data  was  based  on  current  end  items  in  fonnal  AWP  status.  Some  end  items  had 
multiple  parts  backordered  against  it.  This  made  it  impossible  to  apply  the  measurement 
method  prescribed  in  section  4.2.2.  To  account  for  this,  I  treated  each  backordered  part 
as  a  separate  delay,  even  if  it  was  one  of  many  parts  causing  the  delay  for  that  particular 
end  item.  Since  this  data  is  not  being  used  to  make  operational  decisions,  there  is  no 
hann  in  doing  this.  Finally,  an  inherent  limitation  to  using  the  Spares  Priority  Release 
Sequence  priority  is  that  it  can  change  daily.  As  a  result,  the  Spares  Priority  Release 
Sequence  priority  assigned  to  each  end  item  was  based  on  data  pulled  from  EXPRESS  on 
February  4,  2013. 

Pricing  data  could  not  be  located  for  eight  piece  parts.  The  researcher  identified 
the  20  randomly  selected  piece  parts  prior  to  this  realization,  but  all  of  the  20  parts 
selected  had  the  necessary  data.  Therefore,  the  missing  data  had  no  effect  on  the  analysis 
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using  the  Spearman’s  significance  test.  However,  this  could  have  affected  the  outcome 
for  the  practical  comparisons  since  I  ranked  the  entire  data  population.  While  none  of  the 
eight  parts  were  one  of  the  20  randomly  selected  parts,  the  eight  piece  parts  missing 
pricing  data  had  to  be  excluded.  The  eight  parts  should  not  have  a  major  effect  on  the 
results  because  the  applicable  values  were  low  for  all  eight  parts:  end  item  sell  prices 
($1,598  -  $152,522),  Spares  Priority  Release  Sequence  priorities  (none  assigned),  piece 
part  quantities  (1  -  8),  and  frequencies  (1-8).  The  AWP  time  on  the  eight  parts  ranged 
from  40  days  to  3,813  days.  However,  even  the  part  with  3,813  AWP  days  would  have 
been  ranked  42  by  AWP  time  across  the  entire  population.  While  the  AWP  time  is 
somewhat  high  for  some  of  the  excluded  parts,  the  effect  on  the  results  and  conclusions 
should  be  minimal  due  to  their  low  values  in  the  other  variables. 

4.2.5  AWP  Impact  Index 

With  the  final  variables  selected  and  the  necessary  data  collected,  the  next  step 
was  to  combine  the  variables  in  such  a  way  as  to  get  the  proper  effect  from  them.  The 
desired  effects  of  each  were  described  in  section  4.2.1  and  validated/clarified  in  sections 
4.2.2  and  4.2.3.  The  researcher  developed  the  following  equation  for  the  AWP  Impact 
Index  based  on  those  desired  effects: 
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Where 


Ip  =  AWP  Impact  Index  by  Piece  Part 
T  =  AWP  Time,  length  of  delay  (days) 


35 


Q  =  Quantity,  shortage  per  delay  (each) 

S  =  SPRS  Priority,  end  item  priority  per  delay 
Ce  =  Sell  Price  of  End  Item  (dollars) 

Cp  =  Purchase  Price  of  Piece  Part  (dollars) 

Fp  =  Number  of  AWP  delays  caused  by  Piece  Part 
P  =  Piece  Part  (1,  2,  3...n) 
n  =  Number  of  piece  parts 
Wf=  Weight  assigned  to  Frequency 
Wx  =  Weight  assigned  to  AWP  Time 
wq  =  Weight  assigned  to  Quantity 
wc  =  Weight  assigned  to  Cost  Ratio 
ws  =  Weight  assigned  to  SPRS  Priority 

The  AWP  Impact  Index  (Ip)  provides  a  value  that  has  no  meaningful  unit  of 
measure.  It  is  only  useful  when  comparing  the  effects  of  different  piece  parts  and  to 
compare  different  AWP  impact  indices.  Each  of  the  variables  has  a  different  scale,  so  the 
raw  data  was  nonnalized.  To  nonnalize  the  data,  I  divided  each  data  point  by  the 
maximum  value  for  that  variable.  I  also  nonnalized  the  weights  by  dividing  each  raw 
weight  by  the  sum  of  the  weights. 

In  accordance  with  the  recommendations  in  sections  4.2.2  and  4.2.3,  as  each  of 
the  variables  increase,  so  does  the  index  value  of  the  piece  part.  Summing  the  AWP 
time,  quantity,  cost  ratio,  and  Spares  Priority  Release  Sequence  priority  for  each  delay 
double  counts  the  frequency  since  there  is  a  separate  variable  for  frequency.  Therefore, 
the  researcher  used  the  average  of  each  variable,  except  frequency.  The  researcher  did 
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not  use  the  average  frequency  because  the  average  frequency  over  one  time  period  is  one. 
By  using  averages,  the  variables  with  consistently  high  values  receive  a  higher  priority 
than  variables  with  a  few  instances  of  high  values.  Normalizing  the  data  and  applying 
weights  to  each  variable  allows  users  to  adjust  the  equation  as  the  importance  of  variables 
change. 

4.2.6  Analysis  of  AWP  Impact  Index  and  Variables 

The  following  four  tests  compare  the  AWP  Impact  Index  rankings  to  the  rankings 
by  each  variable.  These  tests  determine  if  the  rankings  are  different.  They  do  not 
identify  which  prioritization  method  is  best.  To  detennine  the  rankings  by  each  variable, 
the  researcher  summed  the  values  of  each  variable  by  piece  part,  and  then  ranked  them  in 
descending  order  by  the  variable  value.  This  is  based  on  the  assumption  that  depot 
personnel  will  choose  between  using  the  index  rankings  or  the  rankings  of  one  or  more  of 
the  variables.  Depot  personnel  would  compare  the  variables  separately,  so  double 
counting  the  frequency  is  not  a  concern  as  it  was  for  the  AWP  Impact  Index. 

For  the  Spearman’s  test  to  be  valid,  two  conditions  must  be  met.  The  sample 
must  be  randomly  selected  and  the  probability  distributions  of  the  two  variables  being 
compared  must  be  continuous.  The  researcher  selected  a  random  sample  as  described  in 
section  3.3.  The  AWP  Impact  Index  and  the  variables  are  effectively  continuous  because 
the  raw  data  is  normalized.  Table  1  displays  the  results  of  the  fifteen  significance  tests. 
All  cells  with  a  shaded  background  showed  a  statistically  significant  correlation.  The 
researcher  coded  the  AWP  Impact  Index  with  the  letter  I  and  each  variable  with  a  number 
(1,  2,  3,  4,  5)  that  also  corresponds  to  its  relative  importance. 
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Table  1.  Spearman’s  Rank  Correlation  Test  Results 


Spearman's  Rank  Correlation  Coefficients  (r5),  n=20,  a  =  0.005,  rS  a^  =  0.612 

Variables 

AWP  Impact  Index 
(1) 

Frequency 

(D 

AWP  Time 
(2) 

Quantity 

(3) 

Cost  Ratio 
(4) 

SPRS  Priority 
(5) 

AWP  Impact  Index  (1) 

0.32 

0  65 

049 

0.50 

0.32 

Frequency  (1) 

- 

- 

0  65 

0  90 

0.16 

1.00 

AWP  Time  (2) 

- 

- 

- 

0  64 

0.01 

0.65 

Quantity  (3) 

- 

- 

- 

- 

020 

0.90 

Cost  Ratio  (4) 

- 

- 

- 

- 

- 

0.16 

SPRS  Priority  (5) 

- 

- 

- 

- 

- 

- 

Some  of  the  correlations  are  not  surprising.  For  example,  the  researcher  expected 
that  frequency  and  AWP  time  would  be  correlated  since  AWP  time  is  the  sum  of  the 
AWP  time  for  each  delay.  Also,  quantity  and  frequency  were  expected  to  be  correlated 
because  50%  of  the  sample  had  a  quantity  of  one  and  55%  of  the  sample  had  a  frequency 
of  one.  For  the  same  reason,  quantity  and  AWP  time  were  highly  correlated.  Spares 
Priority  Release  Sequence  priority  was  highly  correlated  with  frequency,  AWP  time,  and 
quantity  because  only  one  part  in  the  sample  caused  a  delay  on  an  end  item  that  had  a 
Spares  Priority  Release  Sequence  priority  assigned  to  it.  As  a  result,  the  Spares  Priority 
Release  Sequence  priority  and  frequency  were  perfectly  correlated.  Finally,  AWP  Impact 
Index  and  AWP  time  were  highly  correlated  (rs  =  0.65). 

The  practical  comparison  to  determine  differences  produced  results  similar  to  the 
Spearman  method.  The  coding  scheme  is  the  same  as  used  in  Table  1.  Table  2  shows 
how  the  AWP  Impact  Index  and  each  variable  ranked  the  same  random  20  piece  parts 
within  the  entire  population  of  data  (n  =  607).  Table  3  shows  the  differences  between  the 
AWP  Impact  Index  rankings  and  the  rankings  by  each  variable.  Table  4  shows  the 
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absolute  values  of  the  differences  displayed  in  Table  3  and  the  sums  of  the  absolute 
differences. 


Table  2.  Rankings  across  the  Data  Population 


Piece  Part  NSN 

AWP  Impact  Index  (1) 

Frequency  (1) 

AWP  Time  (2) 

Quantity  (3) 

Cost  Ratio  (4) 

SPRS  Priority  (5) 

5961012448488EH 

5 

6.5 

4 

10.5 

270 

5 

1560015823928FL 

11 

60 

50 

78 

53 

101 

1680PH091417F 

35 

93.5 

114 

106 

447 

145 

5342PH099938F 

86 

491.5 

145.5 

497  5 

232 

503.5 

1650001382564 

157 

247  5 

372 

81 

75.5 

280  5 

5961015880476 

223 

491.5 

228.5 

497.5 

458 

503.5 

5365PH100973F 

271 

247.5 

276 

261 

177 

280  5 

1560015894865 

276 

247  5 

294 

261 

518 

280.5 

4730015470079 

344 

491.5 

330.5 

181.5 

309 

503.5 

536001 1651571SX 

345 

324  5 

326 

335.5 

509 

352.5 

1560013523948FG 

408 

247  5 

353 

261 

355 

280.5 

1680010136374SX 

424 

491.5 

350.5 

497  5 

345 

503.5 

156001 0704977FG 

463 

324.5 

426.5 

335.5 

60 

352.5 

1560015516445 

467 

491.5 

462 

497  5 

344 

503.5 

53300121 96008SX 

476 

491.5 

493 

497.5 

229 

503.5 

1560015655767 

523 

491.5 

532.5 

497  5 

272 

503.5 

5962014962987EK 

524 

491.5 

538.5 

497.5 

243 

503.5 

481 0004637646HS 

526 

491.5 

549.5 

497  5 

134 

503.5 

1 660NCZ084485BQ 

543 

491.5 

545 

497.5 

577 

503.5 

5977003527037EH 

591 

491.5 

589 

497  5 

406 

503.5 

Table  3.  Differences  between  Rankings 


39 


Piece  Part  NSN 

(I.D 

(1.2) 

(1.3) 

(1,4) 

d.5) 

5961012448488EH 

-1.5 

1 

-5.5 

-265 

0 

1 56001 5823928FL 

49 

-39 

-67 

42 

-90 

1680PH091417F 

-58.5 

-79 

-71 

412 

-110 

5342PH099938F 

405.5 

-59.5 

411.5 

-146 

417.5 

1650001382564 

-90.5 

-215 

76 

81.5 

-123.5 

5961015880476 

-268.5 

-5.5 

-274.5 

-235 

-280.5 

5365PH100973F 

23.5 

-5 

10 

94 

-9.5 

1560015894865 

28.5 

-18 

15 

-242 

4.5 

4730015470079 

-147.5 

13.5 

162.5 

35 

-159.5 

536001 1651571SX 

20.5 

19 

9.5 

-164 

-7.5 

1560013523948FG 

160.5 

55 

147 

53 

127.5 

1 68001 01 36374SX 

-67.5 

73.5 

-73.5 

79 

-79  5 

1560010704977FG 

138.5 

36.5 

1275 

403 

110.5 

1560015516445 

-24.5 

5 

-30.5 

123 

-36.5 

5330012196008SX 

-15.5 

-17 

-21.5 

247 

-27.5 

1560015655767 

31.5 

-9.5 

25.5 

251 

19.5 

5962014962987EK 

32.5 

-14.5 

26.5 

281 

20.5 

4810004637646HS 

34.5 

-23.5 

28.5 

392 

22.5 

1 660NCZ084485BQ 

51.5 

-2 

45.5 

-34 

39.5 

5977003527037EH 

99.5 

2 

93.5 

185 

87.5 
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Table  4.  Absolute  Values  of  Differences 


Piece  Part  NSN 

(1.1) 

d.2) 

(1.3) 

(1,4) 

(1,5) 

5961 01244 8488EH 

1.5 

1 

5.5 

265 

0 

1 56001 5823928FL 

49 

39 

67 

42 

90 

1680PH091417F 

58.5 

79 

71 

412 

110 

5342PH099938F 

405.5 

59.5 

411.5 

146 

417.5 

1650001382564 

90.5 

215 

76 

81.5 

123.5 

5961015880476 

268.5 

5.5 

274.5 

235 

280.5 

5365PH100973F 

23.5 

5 

10 

94 

9.5 

1560015894865 

28.5 

18 

15 

242 

4.5 

4730015470079 

147.5 

13.5 

162.5 

35 

159.5 

536001 1651571SX 

20.5 

19 

9.5 

164 

7.5 

1560013523948FG 

160.5 

55 

147 

53 

127.5 

1680010136374SX 

67.5 

73.5 

73.5 

79 

79.5 

1560010704977FG 

138.5 

36.5 

127.5 

403 

110.5 

1560015516445 

24.5 

5 

30.5 

123 

36.5 

5330012196008SX 

15.5 

17 

21.5 

247 

27.5 

1560015655767 

31.5 

9.5 

25.5 

251 

19.5 

5962014962987EK 

32.5 

14.5 

26.5 

281 

20.5 

4810004637646HS 

34.5 

23.5 

28.5 

392 

22.5 

1660NCZ084485BQ 

51.5 

2 

45.5 

34 

39.5 

5977003527037EH 

99.5 

2 

93.5 

185 

87.5 

SUM 

1749.5 

693 

1722 

3764.5 

1773.5 

Table  4  shows  that  the  sum  of  absolute  differences  is  smallest  between  the  AWP 
Impact  Index  and  the  AWP  time  rankings,  which  indicates  AWP  time  could  have  too 
much  influence  on  the  AWP  Impact  Index.  This  supports  the  results  found  with 
Spearman’s  method  in  Table  1.  However,  adjusting  the  weights  could  mitigate  this  effect 
if  the  users  deem  it  undesirable. 

Tables  5  and  6  show  the  results  of  the  practical  comparison  to  determine 
commonalities.  Table  5  shows  which  piece  parts  were  ranked  in  the  top  20  by  the  AWP 
Impact  Index  and  each  variable.  The  second  vertical  column  in  Table  6  shows  how  many 
times  a  piece  part  from  the  AWP  Impact  Index’s  top  20  made  it  into  the  top  20  of  the 
other  variables’  rankings.  The  bottom  row  in  Table  6  shows  how  many  times  a  variable 
ranked  the  same  part  within  the  top  20  as  the  AWP  Impact  Index.  Finally,  it  shows  the 
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population  rankings  of  each  piece  part  by  each  variable.  If  a  variable  ranked  the  piece 
part  in  the  top  20,  the  cell  for  that  piece  part  is  shaded.  The  results  in  Table  6  support  the 
findings  in  the  previous  two  methods.  The  AWP  Impact  Index  and  AWP  time  the  highest 
number  of  commonalities,  which  suggests  AWP  time  has  an  overpowering  effect  on  the 
AWP  Impact  Index  rankings.  Again,  adjusting  weights  is  a  mitigation  option. 


Table  5.  Top  20  Piece  Parts  by  AWP  Impact  Index  and  each  Variable 


Rank 

AWP  Impact  Index  (1) 

Frequency  (1) 

AWP  Time  (2) 

Quantity  (3) 

Cost  Ratio  (4) 

SPRS  Priority  (5) 

1 

6105003175643SX 

6105003175643SX 

6105003175643SX 

6105003175643SX 

4010012848086 

6105003175643SX 

2 

5331013608012BQ 

1560PH100424F 

1560002133595FL 

1 560PH1 00398F 

5331013608012BQ 

5360003361029SX 

3 

4730PH095404F 

1 5600021 33595FL 

6105007838732SX 

5331013608012BQ 

5330014353459SX 

1660011244775 

4 

15600021 33595FL 

1560PH100398F 

1560015823928FL 

1560PH100424F 

5306PH099180F 

1560001827145FL 

5 

5961012448488EH 

304001 1 830215JF 

1660012075403 

1560002133595FL 

1560014586936SX 

1560015823928FL 

6 

3010002749624UC 

1680012471806RK 

3110000791181 

5306PH0991 80F 

5306PH099179F 

1660015266169 

7 

4010012848086 

1560015823928FL 

1660011244775 

3110000791181 

5330005914669 

2915012078715SX 

8 

6150011617564 FIS 

4810013884700TP 

3110009459317 

3040011 83021 5JF 

1560015436165FL 

293501 1938067PN 

9 

3040010304435AW 

1660015266169 

4810013884700TP 

5340010836689SX 

1560007662589FL 

6620000865223 

10 

1660011244775 

1660012075403 

5930011799694EK 

1680012471806RK 

5365000646283SX 

2915012037050PR 

11 

1560015823928FL 

61 0501232831 8NT 

4730PH095404F 

1560015823928FL 

5310012926334SX 

2915009074704SX 

12 

6105007838732SX 

6105007838732SX 

5310003307757SX 

4810013884700TP 

1 56000941 1214FL 

3120012455083FX 

13 

3040015908392 

3110000791181 

1560PH100424F 

1660015266169 

5325PH085666F 

5325PH085666F 

14 

1560010390680AW 

1650001811760 

5945010441702TP 

28400 12370594PN 

5310000799661SX 

1680010036945SX 

15 

61 05004841 890HY 

1660008664250 

1 6500121 08535HS 

1660012075403 

5340010836689SX 

291501 1827053JF 

16 

1660012075403 

6150001789569VH 

2915012037050PR 

6105012328318NT 

9905012461514FX 

5330013427775SX 

17 

1680015954309 

1 650001 339481 HS 

1560011 900597FG 

5306PH099179F 

5330013613149FW 

4810004609188SX 

18 

3110000791181 

6680002203265UC 

304001242661 9HS 

6105007838732SX 

5340007986885HS 

4810010553406SX 

19 

5945010441702TP 

1560016032651 

1650001811760 

1650001811760 

1560006519167FL 

1660015267017 

20 

1560PH100424F 

1660005225938 

61 0501232831 8NT 

1660008664250 

1560015453064 

9905012461514FX 
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Table  6.  Summary  of  Commonalities  across  Top  20 


Piece  Part  NSN 

#  of  Top  20  in 
Common  with  AWP 
Impact  Index  Top  20 

Rankinc 

is 

AWP 
Impact 
Index  (1) 

Frequency 

(1) 

AWP  Time 
(2) 

Quantity 

(3) 

Cost 
Ratio  (4) 

SPRS 

Priority 

(5) 

6105003175643 SX 

4 

1 

1 

1 

1 

80 

1 

5331013608012BQ 

2 

2 

184 

367 

3 

2 

237  5 

4730PH095404F 

1 

3 

81 

11 

46 

25 

127 

1 5600021 33595FL 

3 

4 

3 

2 

5 

23 

26 

5961012448488EH 

0 

5 

491.5 

145 

497.5 

232 

503.5 

301 0002749624UC 

0 

6 

491.5 

146 

497.5 

462 

503.5 

4010012848086 

1 

7 

247.5 

144 

106 

1 

280.5 

6150011617564HS 

0 

8 

247.5 

64 

261 

516 

280  5 

304001 0304435AW 

0 

9 

491.5 

204 

497.5 

269 

78  5 

1660011244775 

2 

10 

25 

7 

32.5 

319 

3 

1 56001 5823928FL 

4 

11 

6.5 

4 

10.5 

270 

5 

61 05007838732 SX 

3 

12 

13 

3 

18.5 

408 

51 

3040015908392 

0 

13 

491.5 

191 

497.5 

527 

503.5 

1 56001 0390680AW 

0 

14 

213 

52 

216.5 

379 

237.5 

61 05004841 890HY 

0 

15 

101 

28 

126.5 

522 

162 

1660012075403 

3 

16 

10 

5 

14.5 

231 

45 

1680015954309 

0 

17 

324.5 

124 

335.5 

141 

37 

3110000791181 

3 

18 

13 

6 

7 

57 

51 

594501 0441 702TP 

1 

19 

54.5 

14 

51.5 

443 

95.5 

1560PH100424F 

3 

20 

2 

13 

4 

107 

25 

#  of  Top  20  in  Common  with  AWP  Impact  Index 
Top  20 

7 

10 

8 

2 

3 

The  third  comparison  used  quintile  agreement  to  show  how  the  ranking  methods 
compared  across  the  population  of  piece  parts.  In  Figures  3-7,  the  vertical  axis  shows 
how  many  times  the  AWP  Impact  Index  and  the  variables  ranked  the  same  piece  part  in 
the  same  quintile  for  each  comparison,  and  the  horizontal  axis  indicates  the  quintile.  The 
bar  charts  indicate  a  higher  amount  of  agreement  in  the  first  and  fifth  quintiles.  This 
could  be  the  result  of  piece  parts  with  extreme  values  (high  or  low)  consistently  driving 
those  piece  parts  into  the  highest  and  lowest  rankings  regardless  of  the  ranking  method. 
The  charts  also  show  that  AWP  time  had  higher  agreement  regardless  of  quintile 
compared  to  the  other  four  variables.  This  supports  the  findings  in  the  previous  three 
comparison  methods. 
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Figure  3.  AWP  Impact  Index  and  Frequency  (1)  Quintile  Agreement 
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Figure  4.  AWP  Impact  Index  and  AWP  Time  (2)  Quintile  Agreement 
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Figure  5.  AWP  Impact  Index  and  Quantity  (3)  Quintile  Agreement 
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Figure  6.  AWP  Impact  Index  and  Cost  Ratio  (4)  Quintile  Agreement 
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Figure  7.  AWP  Impact  Index  and  SPRS  Priority  (5)  Quintile  Agreement 


Whether  or  not  the  AWP  Impact  Index  produces  a  better  prioritization  of  piece 
parts  than  the  individual  variables  is  a  determination  that  the  76th  Commodities 
Maintenance  Group  must  make.  Only  they  have  the  ability  to  determine  if  the 
prioritization  reflects  the  true  nature  of  AWP  impact.  To  aid  in  that  decision,  the 
researcher  extracted  some  examples  from  the  data  of  the  major  prioritization  differences. 
They  are  displayed  in  Table  7.  Any  piece  part  ranked  between  1  and  30  is  shaded. 
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Table  7.  AWP  Impact  Index  Extreme  Differences 


Piece  Part  NSN 

AWP  Impact 

Index 

Frequency  (1) 

AWP  Time  (2) 

Quantity  (3) 

Cost  Ratio  (4) 

SPRS 

Priority  (5) 

1560PFI100398F 

0.20 

56 

7216 

112 

374 

56 

481001 3884700TP 

0.18 

36 

10523 

36 

88 

36 

30400118 30215JF 

0.16 

49 

4426 

49 

216 

49 

16800 12471806RK 

0.16 

43 

5686 

43 

357 

43 

1660015266169 

0.14 

35 

2969 

35 

258 

350 

5330014353459SX 

0.11 

9 

1468 

9 

25893 

9 

15600145869 36SX 

0.10 

11 

1288 

17 

20466 

11 

5 3400108 36689SX 

0.10 

10 

563 

45 

4995 

10 

1560007662589FL 

0.09 

3 

307 

3 

9988 

3 

1560015436165FL 

0.08 

7 

686 

7 

15040 

7 

1560009411214FL 

0.07 

9 

938 

9 

8472 

9 

1560015453064 

0.06 

2 

202 

2 

3074 

2 

5 3601448 3200 3SX 

0.06 

4 

134 

4 

331 

61 

1560015518617 

0.05 

2 

187 

2 

2005 

2 

Table  7  shows  how  the  AWP  Impact  Index  produces  prioritizations  that  contradict  the 
prioritizations  of  the  most  important  variables  identified  by  the  expert  panel.  Most  of  the 
differences  are  between  cost  ratio  rankings  and  all  other  variables’  rankings.  These 
differences  appear  to  cause  some  piece  parts  (i.e.-the  second,  third,  fourth,  and  fifth  listed 
in  Table  7)  to  not  be  ranked  in  the  top  30  by  the  AWP  Impact  Index,  despite  two  or  more 
variables  ranking  them  in  the  top  30.  These  contradictions  must  be  considered  if  the 
AWP  Impact  Index  is  to  be  implemented  by  the  depot.  However,  another  important 
detail  to  recognize  is  the  high  amount  of  agreement  among  frequency,  AWP  time,  and 
quantity.  While  these  are  not  differences,  subject  matter  experts  should  examine  the 
similarities  as  well  to  determine  if  the  parts  are  ranked  properly.  Subject  matter  experts 
in  the  76th  Commodities  Maintenance  Group  should  examine  the  examples  in  Table  7  to 
determine  if  such  rankings  are  appropriate.  In  addition,  subject  matter  experts  can  use 
such  comparisons  to  detennine  whether  or  not  change  the  weights  assigned  to  variables. 
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4.3  Gap  Analysis  Results 

Prior  to  the  on-site  data  collection,  the  researcher  collected  infonnation  about  the 
general  repair  process  via  email  and  phone  discussions.  This  was  done  to  limit  the 
amount  of  on-site  time  required  and  to  ensure  on-site  discussions  were  as  productive  and 
focused  as  possible.  Narrowing  the  researcher’s  knowledge  gap  prior  to  the  on-site  visit 
was  absolutely  crucial  to  knowing  what  questions  to  ask,  but  more  importantly,  to  know 
what  terminology  to  use  so  questions  would  elicit  accurate,  factual  responses.  All  on-site 
information  gathering  used  the  flow  chart  in  Figure  2  as  a  basis. 

4.3.1  Fuels  Accessories  Shop 

The  first  shop  visited  was  the  fuels  accessories  shop.  The  scheduler  first  provided 
an  overview  of  the  repair  process  of  the  fuels  accessories  shop.  Then,  the  researcher 
conducted  a  physical  walk  through  of  the  process  to  ensure  accuracy  of  the  infonnation 
received  by  the  scheduler.  A  mechanic  also  provided  details  about  the  Disassembly  and 
Repair  &  Assembly  steps.  In  addition,  the  technicians  who  perfonn  the  Kitting  step 
provided  details  about  their  work.  The  process  is  depicted  in  Figure  8.  Kitting  is 
depicted  with  an  asterisk  because  not  all  end  items  go  through  the  Kitting  step. 
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Figure  8.  Fuels  Accessories  Shop  Repair  Process  Flow  Chart  (Current  State) 


The  repair  process  in  the  fuels  accessories  shop  starts  with  end  items  needing 
repaired  being  “pushed”  to  the  shop  by  EXPRESS.  EXPRESS  makes  this  determination 
based  on  many  variables  including  demand  from  the  field  and  piece  parts  availability. 

See  section  2.2  for  a  more  detailed  explanation  of  the  induction  logic  EXPRESS  uses.  If 
all  requirements  are  met,  then  the  end  item  is  “pushed”  to  the  shop  for  repair.  Upon 
receipt,  the  scheduler  inducts  the  end  item.  The  scheduler  prints  a  Work  Control 
Document  (WCD),  which  has  three  bar  codes  on  it  for  each  step  in  the  repair  process: 
Start,  Complete,  and  Delay.  The  scheduler  scans  the  Start  bar  code,  which  is  recorded  in 
the  Time,  Attendance,  and  Accountability  (TAA)  system. 
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TAA’s  primary  purpose  is  to  maintain  accountability  of  worker’s  hours,  time 
spent  on  each  step,  and  other  miscellaneous  purposes.  The  information  recorded  via  the 
Time,  Attendance,  and  Accountability  system  feeds  into  multiple  other  systems, 
including  the  Inventory  Tracking  System  (ITS).  Only  certain  pieces  of  infonnation 
recorded  through  the  Time,  Attendance,  and  Accountability  system  are  sent  to  each  of  the 
other  systems.  The  information  pertinent  to  this  research  is  not  stored  in  the  Time, 
Attendance,  and  Accountability  system;  therefore,  it  is  considered  a  front-end 
information  system  where  the  pertinent  information  is  entered  by  the  technician,  but 
stored  elsewhere.  In  this  case  the  pertinent  information  (starts,  completions,  and  delays) 
is  stored  in  the  Inventory  Tracking  System. 

Once  the  scheduler  has  inducted  the  end  item,  he/she  scans  Complete  if  a 
mechanic  is  available  with  the  proper  skill  set.  Otherwise,  the  scheduler  scans  the  Delay 
bar  code.  If  the  Delay  bar  code  is  scanned,  the  scheduler  must  enter  a  delay  code.  A 
drop  down  menu  of  the  delay  codes  appears  on  the  screen  in  the  Time,  Attendance,  and 
Accountability  system.  There  are  more  than  a  dozen  delay  codes  that  can  be  entered, 
which  cover  a  wide  variety  of  reasons  for  delays.  Most  of  the  time,  at  this  step,  the 
scheduler  selects  the  RO 1  delay  code,  which  indicates  that  work  cannot  continue  on  the 
end  item  because  the  personnel  or  skill  set  needed  is  unavailable. 

After  induction,  approximately  75-80%  of  end  items  move  on  to  the  Kitting  step 
where  any  piece  parts  with  an  80%  probability  of  needing  replaced  are  ordered.  The  end 
item  does  not  leave  Kitting  until  all  ordered  piece  parts  have  been  received.  As  a  result, 
this  is  the  first  step  where  AWP  delays  can  occur.  The  Kitting  step  was  added  by  the 
76th  Commodities  Maintenance  Group  to  prevent  end  items  with  historically  long  AWP 
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delays  from  clogging  up  the  repair  process.  This  is  necessary  because  the  depots  do  not 
control  the  flow  of  end  items  into  their  shops.  EXPRESS  dictates  what  gets  repaired  and 
what  doesn’t.  The  Kitting  step  is  an  effort  by  the  76th  Commodities  Maintenance  Group 
to  minimize  queues  in  the  repair  process.  However,  it  appears  this  step  simply  shifts  the 
location  of  the  queues  into  the  Kitting  step  because  end  items  in  Kitting  do  not  receive  a 
higher  priority  than  any  other  end  items  in  the  repair  process,  unless  it  is  put  through  the 
formal  AWP  process.  The  priority  of  an  end  item  is  based  on  its  priority  to  the  customer, 
regardless  of  where  it  is  in  the  repair  process.  However,  if  there  are  extra  setup  costs  or 
time  associated  with  starting  repair  work  and  then  stopping  repair  work  due  to  an  AWP 
delay,  this  would  result  in  a  shorter  flow  time  by  putting  the  end  item  through  the  Kitting 
step.  The  researcher  did  not  see  any  indication  of  significant  extra  setup  costs  or  time  in 
this  shop.  However,  this  could  not  be  verified  as  it  was  not  the  focus  of  the  research. 

In  the  Kitting  step,  the  piece  parts  are  ordered  in  the  Automated  Bill  of  Materials 
(ABOM)  system.  The  Automated  Bill  of  Materials  system  is  another  front-end  system 
where  infonnation  is  entered  by  the  technician,  but  stored  elsewhere.  The  information 
entered  in  the  Automated  Bill  of  Materials  system  feeds  into  the  Navair  Industrial 
Material  Management  System  (NIMMS),  where  it  is  stored.  NIMMS  is  primarily  a 
financial  accounting  system,  so  the  Air  Force  can  account  for  the  materials  used  to  repair 
end  items.  However,  it  also  communicates  with  the  D035K,  Wholesale  and  Retail 
Receiving  and  Shipping  (WARRS)  system,  which  is  one  of  the  supply  systems  used  by 
the  Air  Force  (Air  Force  Materiel  Command,  2006:  52).  As  a  result,  the  infonnation 
entered  in  the  Automated  Bill  of  Materials  system  is  stored  in  two  infonnation  systems 
(NIIMS  and  D035K).  The  time  an  end  item  spends  in  the  Kitting  step  is  not  recorded 
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through  the  Time,  Attendance,  and  Accountability  system  as  it  is  in  the  Induction  step. 
However,  the  time  an  end  item  spends  in  the  Kitting  step  can  be  captured  by  measuring 
the  difference  between  the  time  Induction  was  completed  in  the  Time,  Attendance,  and 
Accountability  system  to  the  time  Disassembly  was  started  in  TAA. 

The  next  step  is  Disassembly.  For  any  end  items  that  do  not  go  through  Kitting, 
Disassembly  is  accomplished  right  after  Induction.  In  this  step,  end  items  are 
disassembled,  cleaned,  and  inspected.  Just  as  with  the  scheduler,  the  mechanic  uses  the 
three  bar  codes  on  the  Work  Control  Document  to  record  starts,  completions,  and  delays 
in  the  Time,  Attendance,  and  Accountability  system  throughout  this  step.  During  this 
step,  repair  or  replace  determinations  are  made  for  piece  parts.  If  a  piece  part  needs  to  be 
replaced,  the  order  is  placed  in  SKIL  (Scheduling  and  Kitting  Inventory  Listing)  by  the 
mechanic.  SKIL  is  a  system  created  by  and  used  only  at  Tinker  AFB.  It  communicates 
with  multiple  other  information  systems,  including  the  Automated  Bill  of  Materials 
system,  which  is  why  piece  parts  are  ordered  through  the  Scheduling  and  Kitting 
Inventory  Listing.  If  piece  parts  are  backordered  in  this  step,  it  does  not  delay  the  end 
item.  The  end  item  will  be  moved  on  to  the  Repair  and  Assembly  step  regardless  of  the 
status  of  piece  parts  ordered. 

The  end  item  then  moves  to  the  Repair  and  Assembly  step.  At  this  time, 
mechanics  begin  repairing  or  replacing  piece  parts  as  necessary  and  assembling  the  end 
item  as  they  progress  through  the  repairs.  Again,  the  Time,  Attendance,  and 
Accountability  system  is  used  to  record  the  starts,  completions,  and  delays  throughout 
this  step.  This  is  the  second  and  final  step  where  AWP  delays  can  occur.  If  the  mechanic 
does  come  to  a  work  stoppage  because  the  part  or  parts  needed  are  not  available,  he/she 
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scans  the  delay  bar  code  and  enters  the  appropriate  delay  code.  In  some  cases,  the  parts 
may  not  be  available  because  status  on  them  has  not  yet  been  received.  This  most  often 
occurs  early  in  the  step  when  orders  have  just  recently  been  placed.  The  delay  code  for 
this  scenario  is  R05.  However,  if  the  mechanic  finds  out  that  the  part(s)  needed  is 
backordered,  the  R06  delay  code  is  selected.  Therefore,  the  start  of  AWP  delays  are 
already  captured  using  the  R06  delay  code. 

The  end  item  is  then  set  aside  and  the  mechanic  will  begin  work  on  other  end 
items.  The  exception  to  this  is  if  the  end  item  needs  to  be  placed  in  formal  AWP.  There 
is  a  separate  process  where  end  items  can  be  placed  in  formal  AWP  status,  which  gives 
the  parts  backordered  for  that  end  item  a  higher  priority  over  end  items  that  are  not  in 
formal  AWP.  If  an  end  item  is  put  into  formal  AWP  status,  then  it  is  removed  from  the 
shop  and  locked  in  a  cage  controlled  by  the  supplier  (the  Defense  Logistics  Agency  in 
most  cases).  The  end  item  stays  in  that  cage  until  all  backordered  parts  are  received.  It  is 
then  brought  back  to  the  shop.  Whether  in  fonnal  AWP  status  or  just  set  aside  in  the 
shop,  the  R06  delay  code  stays  in  effect  until  a  mechanic  scans  in  another  status,  which 
could  be  Start  or  Delay  with  a  different  delay  code  selected.  In  some  cases,  the  mechanic 
identifies  parts  that  need  replaced  and  were  not  ordered  in  the  Disassembly  step.  This 
could  be  a  result  of  an  oversight,  a  defective  part  was  received,  the  part  was  damaged 
during  installation,  etc.  Regardless  of  the  reason,  the  piece  part  needed  is  ordered 
through  the  Scheduling  and  Kitting  Inventory  Listing  system,  just  as  it  is  done  in  the 
Disassembly  step. 

Once  all  repairs  are  completed,  the  end  item  is  moved  to  the  Test  step.  This  is 
where  all  end  items  are  tested.  If  any  end  item  fails  during  the  Test  step,  personnel  in  the 
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Test  step  send  it  back  to  the  Repair  and  Assembly  step  for  the  necessary  corrective 
action,  then  sent  back  to  Test.  As  a  result,  no  AWP  delays  can  occur  in  the  Test  step.  If 
a  piece  part  is  the  cause  of  a  failure  in  the  Test  step  and  needs  to  be  replaced,  the  end  item 
is  sent  back  to  the  Repair  and  Assembly  step  for  the  necessary  repairs.  This  process  is 
repeated  until  the  end  item  passes  the  Test  step.  Finally,  the  end  item  is  “sold”  to  the 
customer. 

4.3.2  Governors  and  Accessories  Shop 

The  second  shop  visited  was  the  Governors  and  Accessories  Shop.  For  the  sake 
of  brevity,  the  entire  process  will  not  be  described.  The  process  followed  in  this  shop  is 
very  similar  to  the  Fuel  Accessories  Shop,  so  only  the  exceptions  to  the  process  described 
in  section  4.3.1  will  be  detailed  here.  Figure  9  depicts  the  repair  process  followed  by  the 
Governors  and  Accessories  Shop. 
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Figure  9.  Governors  and  Accessories  Shop  Repair  Process  Flow  Chart  (Current  State) 


Induction  is  accomplished  just  as  described  in  section  4.3.1,  to  include  using  the 
same  information  systems.  However,  the  Kitting  step  is  not  accomplished  for  any  end 
items  repaired  in  this  shop.  It  is  being  discussed  as  an  option  for  this  shop,  but  has  not 
yet  been  implemented.  The  Disassembly  and  Repair  and  Assembly  steps  are 
accomplished  in  the  same  manner  as  described  in  section  4.3.1,  with  one  exception.  If 
any  piece  parts  are  backordered,  the  end  item  is  immediately  put  through  the  formal 
AWP  process.  The  R06  delay  code  is  still  entered  in  the  Time,  Attendance,  and 
Accountability  system,  so  the  AWP  delay  is  recorded.  The  reason  for  this  practice  is  due 
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to  the  long  waits  commonly  experienced  for  backordered  piece  parts  in  this  shop.  There 
are  no  noteworthy  differences  for  the  Test  and  Sell  steps. 

4.3.3  Constant  Speed  Drives  (CSD)  Shop 

Again  for  the  sake  of  brevity,  only  exceptions  to  the  process  described  in  section 
4.3.1  will  be  discussed.  Figure  10  shows  the  current  state  of  the  repair  process  for  the 
Constant  Speed  Drives  shop. 


Figure  10.  CSD  Shop  Repair  Process  Flow  Chart  (Current  State) 


The  first  difference  of  note  is  that  Kitting  does  not  have  an  asterisk  next  to  it.  This  is 
because  all  end  items  repaired  in  the  Constant  Speed  Drives  shop  go  through  the  Kitting 
step.  This  is  done  because  of  the  historically  long  AWP  delays.  It  is  also  worth 
reiterating  that  the  delays  experienced  in  the  Kitting  step  are  not  recorded  in  the  Time, 
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Attendance,  and  Accountability  system.  There  are  no  differences  in  the  Disassembly 
step.  Due  to  the  nature  of  the  end  items  repaired  in  the  Constant  Speed  Drives  shop, 
subassemblies  (children  parts)  of  the  end  items  are  sent  to  other  shops  in  the  Oklahoma 
City  Air  Logistics  Complex.  These  subassemblies  are  only  repaired  in  the  other  shops 
and  do  not  have  piece  parts  replaced  in  them,  except  those  manufactured  by  the  shops.  In 
other  words,  if  a  piece  part  needs  to  be  replaced,  it  is  fabricated  in  that  shop  and  is  not 
dependent  on  an  outside  source  of  supply.  Once  all  repairs  are  completed,  the 
subassemblies  are  returned  to  the  Constant  Speed  Drives  shop.  After  all  subassemblies 
are  returned  to  the  CSD  shop  for  an  end  item,  the  Repair  and  Assembly  step  is  started. 
This  step  and  all  remaining  steps  are  perfonned  as  described  in  section  4.3.1. 

4.3.4  Overall  Repair  Process 

To  simplify  the  gap  analysis,  these  three  shops’  processes  were  generalized  and 
combined  into  one  overall  repair  process,  depicted  in  Figure  11. 
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Figure  1 1 .  Overall  Repair  Process  Flow  Chart  (Current  State) 


The  process  flow  chart  in  Figure  1 1  is  the  same  as  depicted  in  Figure  8.  This  is  possible 
because  the  Kitting  step  has  an  asterisk  next  to  it  indicating  that  not  all  end  items  process 
through  the  Kitting  step,  which  is  still  a  true  statement.  The  Repair  of  Children  Parts  step 
from  the  Constant  Speed  Drives  shop  was  removed  from  the  flow  chart  because  it  can 
just  as  easily  be  explained  as  a  sub-step  within  the  Repair  and  Assembly  step. 
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The  locations  of  possible  AWP  delays  in  the  repair  process  are  identified  in 


Figure  12. 


Figure  12.  Possible  AWP  Delays 


4.2.5  Desired  End  State 

The  desired  end  state  should  enable  the  creation  of  a  list  of  piece  parts  that  cause 
AWP  delays.  Since  there  are  only  two  steps  in  the  repair  process  where  AWP  delays  are 
possible,  the  information  regarding  AWP  delays  would  only  need  to  be  recorded  in  those 
two  steps.  In  addition,  this  information  should  be  recorded  in  one  infonnation  system  to 
ensure  ease  of  use.  The  information  included  in  this  list  based  on  the  variables  identified 
in  section  4.2.  First,  the  National  Stock  Number  (NSN)  of  the  piece  part  should  be 
recorded  any  time  it  causes  a  delay,  so  that  the  part  can  be  uniquely  identified.  Second, 
the  piece  part  document  number  should  be  recorded,  so  the  order  for  the  piece  part  can  be 
uniquely  identified.  Third,  the  end  item  national  stock  number  should  be  identified 
anytime  it  is  delayed.  Fourth,  the  end  item  document  number  should  be  recorded,  so  each 
delay  can  be  uniquely  identified.  Fifth,  the  start  and  stop  time  of  the  delay,  by  piece  part 
document  number  and  end  item  document  number,  should  be  recorded,  so  the  total  AWP 
time  for  that  specific  delay  can  be  calculated.  Sixth,  the  quantity  ordered  should  be 
recorded,  so  the  magnitude  of  the  shortage  can  be  considered.  Seventh,  the  frequency 
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needs  to  be  recorded  and  can  be  calculated  based  on  the  information  already  listed  above. 
Eighth,  the  sell  price  of  the  delayed  end  item  needs  to  be  recorded.  Ninth,  the  purchase 
price  of  the  piece  part  causing  the  delay  should  be  recorded.  Finally,  the  Spares  Priority 
Release  Sequence  priority  associated  with  the  end  item  delayed  needs  to  be  recorded. 

This  infonnation  should  be  recorded  wherever  an  AWP  delay  can  occur.  Figure  13 
shows  the  desired  end  state  of  information  that  should  be  recorded  in  the  applicable  steps. 
Kitting  is  shown  with  an  asterisk  to  indicate  that  not  all  end  items  are  put  through  the 
Kitting  step. 


Figure  13.  Desired  End  State 


4.2.6  The  Gaps 

With  the  desired  end  state  and  the  current  state  of  the  repair  process  defined,  the 
gaps  can  be  identified.  In  this  gap  analysis,  the  only  gaps  applicable  are  with  regard  to 
AWP  delays.  Therefore,  only  two  steps  need  to  be  examined  for  gaps:  Kitting  and  Repair 
and  Assembly.  To  simplify  the  comparison,  the  six  information  systems  currently  used 
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to  record  and/or  store  data  regarding  AWP  delays  were  consolidated  into  one  depiction. 
In  Figure  14,  this  consolidated  depiction  is  labeled  “Current  State”  and  it  is  shown  below 
the  “Desired  End  State”  information  requirements.  This  allows  for  a  direct  comparison 
of  the  current  state  of  information  being  recorded  and  the  desired  state  of  information 
recorded. 


Figure  14.  Comparison  of  Current  State  and  Desired  End  State 


Overall,  it  appears  that  most  of  the  required  information  is  already  being  recorded. 
However,  the  end  item  sell  price,  piece  part  purchase  price,  and  Spares  Priority  Release 
Sequence  priority  are  not  recorded  in  the  information  systems  identified  in  Figure  14.  It 
also  appears  that  the  frequency  is  not  currently  recorded,  which  is  partially  correct. 
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Figure  11,  “Overall  Repair  Process  Flow  Chart  (Current  State),”  shows  that  the  frequency 
of  delays  can  be  detennined  for  any  end  items  that  experience  an  AWP  delay  in  the 
Repair  and  Assembly  step  because  every  delay  is  recorded  and  an  AWP  delay  is  recorded 
as  an  R06  delay,  which  indicates  the  delay  was  due  to  parts  unavailability. 

In  addition,  each  end  item  that  processes  through  the  Kitting  step  is,  by  definition, 
experiencing  an  AWP  delay,  so  the  number  of  end  items  processing  through  the  Kitting 
step  indicates  the  frequency  of  AWP  delays.  However,  in  the  Kitting  step,  the  start  and 
stop  time  of  any  piece  part  causing  a  delay  can  only  be  ascertained  based  on  the  date/time 
a  piece  part  officially  goes  into  and  out  of  a  backordered  status  in  the  Navair  Industrial 
Material  Management  System  or  the  D035K  system.  This  does  not  capture  the  time 
between  a  piece  part  not  being  available  in  the  shop  and  it  being  placed  into  backorder 
status.  Also,  it  does  not  capture  the  time  between  a  part  coming  out  of  backorder  status 
and  the  part  physically  arriving  in  the  shop.  It  is  a  subtle  difference,  but  can  account  for 
as  little  as  one  day  or  ten  days.  Therefore,  its  impact  is  of  practical  significance. 

Both  states  in  Figure  14  show  that  AWP  delays  are  being  recorded.  However,  in 
the  current  state,  the  delay  is  not  directly  attributed  to  the  piece  part(s)  causing  it.  The 
cause  of  the  delay  can  be  ascertained  by  searching  in  the  Navair  Industrial  Material 
Management  System  or  the  D035K  system  for  the  piece  parts  that  were  backordered  at 
the  time  of  the  delay.  Without  linking  the  delay  to  the  specific  piece  parts  causing  it,  the 
list  of  piece  parts  causing  delays  cannot  be  created. 

In  addition  to  the  comparison  of  what  information  is  recorded  in  the  two  steps, 
there  must  also  be  a  comparison  of  where  the  information  is  recorded.  In  the  desired  end 
state,  all  information  about  AWP  delays  is  recorded  in  one  infonnation  system. 
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However,  as  shown  in  Figure  14,  the  information  is  being  recorded  in  six  different 
information  systems.  Specifically,  in  the  Kitting  step,  the  infonnation  is  being  recorded 
in  the  Automated  Bill  of  Materials  system,  which  feeds  into  Navair  Industrial  Material 
Management  System,  which  also  feeds  into  the  D035K  system.  From  a  practical 
perspective,  the  information  is  available  in  the  Navair  Industrial  Material  Management 
System  and  the  D035K  system  since  the  Automated  Bill  of  Materials  system  is  only  a 
front-end  system  and  no  information  is  stored  in  ABOM.  The  information  recorded  in 
the  Repair  and  Assembly  step  is  stored  in  the  Inventory  Tracking  System  through  the 
Time,  Attendance,  and  Accountability  system.  TAA  does  not  record  and  maintain  the 
information  of  interest  to  this  research;  it  is  another  front-end  infonnation  system.  As  a 
result,  the  AWP  delays  are  being  recorded  in  Navair  Industrial  Material  Management 
System,  the  D035K  system,  and  the  Inventory  Tracking  System  instead  of  just  one 
information  system.  This  means  that  even  though  most  of  the  necessary  information  is 
currently  being  recorded,  it  is  difficult  to  access  it  from  three  information  systems.  This 
was  the  biggest  gap  identified. 
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V. 


Conclusions 


5.1  Overview 

The  intent  of  this  research  was  to  develop  a  process  for  creating  a  prioritized  list 
of  piece  parts  that  are  having  the  greatest  negative  impact  on  depot  operations,  in  addition 
to  developing  a  measurement  (metric)  by  which  depots  could  prioritize  that  list  of  piece 
parts.  Based  on  the  results  discussed  in  Chapter  4,  such  a  list  is  possible  through  two 
methods.  The  first  method  requires  only  minimal  changes  to  the  infonnation  recorded 
currently  and  could  be  implemented  within  six  months.  The  second  method  would  also 
require  only  minimal  changes  to  the  information  recorded,  but  would  require 
substantially  more  time  and  money  for  software  programming  (roughly  4-6  years).  The 
analysis  of  the  AWP  Impact  Index  indicates  that  it  does  produce  priorities  that  are 
significantly  different  from  its  individual  variables,  except  AWP  time.  However,  further 
research  and  development  of  the  metric  is  needed,  so  the  metric  should  be  used  only  on  a 
limited  basis.  Finally,  opportunities  for  future  research  related  to  this  topic  are  discussed. 

5.2  Developing  the  Piece  Parts  List 

At  first  glance,  the  necessary  information  is  not  available  to  develop  a  prioritized 
list  of  piece  parts  causing  AWP  delays.  However,  to  develop  the  list,  one  has  to  consider 
the  sources  of  the  infonnation  needed.  Who/what  would  have  information  on  AWP 
delays  in  the  repair  process?  The  results  show  that  AWP  delays  can  only  occur  in  the 
Kitting  step  or  the  Repair  &  Assembly  step.  Therefore,  the  individuals  performing  the 
tasks  in  those  steps  are  the  infonnation  sources  for  the  delays. 
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5.2.1  Option  #1 

Currently,  nothing  performed  in  the  Kitting  step  is  recorded  in  the  Time, 
Attendance,  and  Accountability  system.  To  be  consistent  with  the  other  steps  of  the 
repair  process,  personnel  in  the  Kitting  step  should  record  Starts,  Completions,  and 
Delays  in  the  Time,  Attendance,  and  Accountability  system.  This  would  also  allow  the 
depot  to  capture  ALL  of  the  AWP  delays  in  one  infonnation  system.  If  technicians  in  the 
Kitting  step  enter  the  R06  delay  code  in  the  Time,  Attendance,  and  Accountability  system 
when  they  place  orders  for  piece  parts,  then  the  actual  start  of  the  delay  time  could  be 
captured.  Next,  the  end  of  the  delay  time  by  piece  part  needs  to  be  captured.  In  the 
Kitting  step,  technicians  detennine  which  piece  parts  have  an  80%  or  greater  probability 
of  needing  replaced.  The  technicians  then  enter  the  orders  for  those  parts  in  the 
Automated  Bill  of  Materials  system,  which  feeds  into  Navair  Industrial  Material 
Management  System  and  the  D035K  system.  As  a  result,  a  list  of  parts  necessary  for 
repair  of  the  end  item  is  in  the  D035K  system.  However,  the  only  parts  of  interest  are  the 
ones  not  available  to  the  technicians.  In  other  words,  backordered  piece  parts  are  the 
only  parts  of  interest.  The  end  of  the  delay  time  for  each  piece  part  can  be  captured  by 
recording  when  the  piece  part’s  status  in  D035K  changes  from  backordered  (BB,  BC, 

BD,  or  BI)  to  any  other  status  code  (Air  Force  Materiel  Command,  2007:  118-119). 

For  the  Repair  &  Assembly  step,  the  information  source  for  AWP  delays  is  the 
mechanics  working  on  the  end  item.  In  this  step,  the  mechanics  already  record  when  an 
AWP  delay  occurs.  They  record  the  delay  in  the  Inventory  Tracking  System  via  the 
Time,  Attendance,  and  Accountability  system  with  an  R06  delay  code.  However,  they  do 
not  currently  record  which  parts  caused  the  delay.  Logically,  they  should  not  be  waiting 
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on  any  parts  that  are  already  available;  therefore,  they  should  only  be  waiting  on  parts 
that  are  backordered.  The  backordered  parts  for  the  end  item  will  be  in  D035K.  There 
are  exceptions  to  this  scenario.  For  example,  the  necessary  parts  are  in  stock,  but  they 
have  not  yet  arrived  from  their  storage  location.  In  addition,  a  part  could  have  been 
initially  backordered,  fulfilled,  but  still  be  in  transit  from  the  supplier.  However,  even  in 
these  scenarios,  the  technician  is  still  at  a  work  stoppage  and  that  fact  should  be  captured. 
The  event  may  get  recorded  as  zero  delay  time,  but  would  still  be  recorded  as  an  instance 
of  a  work  stoppage.  Also,  one  has  to  keep  in  mind  the  purpose  of  the  parts  list.  It  is 
meant  to  be  used  for  root  cause  analysis  and  problem  solving  efforts,  so  the  scenarios 
described  would  still  need  to  be  considered  in  any  problem  solving  effort. 

So  far,  it  has  been  shown  that  the  information  on  delays  is  available,  or  could  be 
with  only  minor  changes  to  the  Kitting  step.  The  next  problem  to  be  solved  is  how  to 
access  and  sort  the  information.  Based  on  discussions  with  the  information  system 
offices  of  primary  responsibility  (OPRs)  and  programmers  at  Tinker  AFB,  it  appears  that 
Cognos  8  could  retrieve  and  sort  the  desired  information.  Cognos  8  is  “a  SQL  report 
writing  tool  that  can  access  data  from  one  or  more  databases  and  is  the  Air  Force  standard 
for  reporting  tool”  (Grilley,  2013).  Cognos  should  be  able  to  query  the  Inventory 
Tracking  System  for  any  end  items  that  have  an  R06  delay  code  loaded  against  them. 

The  R06  delay  code  indicates  an  end  item  is  at  a  work  stoppage  because  it  is  awaiting 
parts.  Then,  Cognos  could  query  D035K  for  the  parts  backordered  against  those  end 
items.  The  Cognos  report  would  then  be  able  to  display  the  total  delay  time  across  the 
76th  Commodities  Maintenance  Group  by  piece  part.  In  addition,  the  report  could 
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display  the  frequency  of  delays  by  piece  part,  and  the  total  quantity  ordered,  which  is  all 
available  in  the  Inventory  Tracking  System  and  D035K. 

In  order  to  use  the  AWP  Impact  Index,  Cognos  would  also  have  to  collect  data  on 
the  end  item  sell  price,  piece  part  purchase  price,  and  the  Spares  Priority  Release 
Sequence  priority.  The  end  item  sell  price  is  available  in  the  Job  Order  Production 
Master  System  (JOPMS),  which  is  a  “repository  for  storing  the  production  number 
master  records”  (Goodman,  2013).  The  piece  part  purchase  price  and  Spares  Priority 
Release  Sequence  priority  are  available  in  EXPRESS.  Therefore,  Cognos  would  have  to 
cross-reference  the  list  of  end  items  and  piece  parts  pulled  from  the  Inventory  Tracking 
System  and  the  D035K  system,  with  the  Job  Order  Production  Master  System  and 
EXPRESS  to  gain  the  necessary  data  for  the  cost  ratio  and  Spares  Priority  Release 
Sequence  priority,  as  shown  in  Figure  15.  Figure  15  shows  how  Cognos  can  match  end 
item  national  stock  numbers  and  document  numbers  and  piece  part  national  stock 
numbers  and  document  numbers  across  information  systems  to  pull  the  necessary  data 
and  compile  it  in  a  fonnat  defined  by  the  user.  This  has  the  same  effect  of  all  the 
information  being  stored  in  one  information  system.  As  a  result,  the  list  could  then  be 
prioritized  using  the  AWP  Impact  Index  or  any  of  its  variables. 
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Figure  15.  Option  #1  Method  of  Retrieving  Data 


However,  such  a  report  would  present  all  parts  backordered  at  the  time  of  the 
delay.  This  is  useful  for  providing  a  snapshot  of  the  current  delays,  so  that  managers  can 
detennine  which  parts  are  most  needed  at  that  particular  moment.  Based  on  the  expert 
panel’s  recommendations,  only  the  last  piece  part  to  come  in,  thus  ending  the  delay, 
should  be  recorded  as  the  cause  of  the  delay  (see  section  4.2.2).  This  could  be 
accomplished  by  applying  a  logic  function  that  only  records  the  piece  part  with  the  most 
recent  status  change  date  or  the  longest  delay  time  per  occurrence.  However,  this  method 
can  only  be  fully  applied  when  taking  a  historical  view  of  the  data.  For  example,  one 
could  pull  a  Cognos  report  that  displayed  data  for  all  end  items  sold  in  2012,  regardless 
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of  the  induction  date.  This  would  exclude  any  end  items  that  are  currently  delayed,  but  it 
would  allow  the  user  to  attribute  each  delay  to  only  one  piece  part.  Otherwise,  if  a  report 
was  pulled  that  displayed  all  end  items  currently  in  the  repair  process  (meaning  they  have 
been  inducted,  but  not  sold),  each  end  item  that  is  delayed  could  have  multiple  piece  parts 
backordered  against  it  at  the  time  of  the  report,  thus  making  it  impossible  to  attribute  each 
delay  to  only  one  piece  part.  This  is  not  necessarily  a  problem.  It  is  simply  a  fact  that 
must  be  considered  when  requesting  Cognos  reports  to  ensure  that  the  user  gets  the 
proper  data  reported. 

5.2.2  Option  #2 

A  second  option  is  more  costly  in  time  and  money,  but  more  accurate.  Just  as  in 
the  first  option,  technicians  in  the  Kitting  step  would  have  to  start  recording  R06  delays 
in  the  Inventory  Tracking  System  via  the  Time,  Attendance,  and  Accountability  system. 
However,  if  the  programming  money  could  be  acquired,  two  features  could  be  added  to 
the  Time,  Attendance,  and  Accountability  system  to  allow  a  more  detailed  recording  of 
AWP  delays.  First,  whenever  the  R06  delay  code  was  selected  in  the  Time,  Attendance, 
and  Accountability  system,  another  window  should  appear,  which  requires  the  technician 
to  select  which  part(s)  from  the  bill  of  materials  is  causing  the  work  stoppage.  Then, 
whenever  the  technician  scanned  the  Start  bar  code  again,  if  it  was  in  Delay  status  with 
an  R06  delay  code,  the  bill  of  materials  window  should  appear  again  to  require  the 
technician  to  select  which  part(s)  allowed  them  to  begin  work  again.  This  would  be 
repeated  anytime  the  technician  experienced  an  AWP  delay.  If  that  information  was 
recorded  in  the  Time,  Attendance,  and  Accountability  system,  then  it  would  feed  into  the 
Inventory  Tracking  System  and  all  the  data  needed  to  identify  and  track  AWP  delays 
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would  be  available  in  the  Inventory  Tracking  System.  Just  as  described  in  section  5.2.1, 
a  Cognos  report  would  need  to  pull  the  necessary  data  from  the  Inventory  Tracking 
System,  Job  Order  Production  Management  System,  and  EXPRESS,  in  order  to  prioritize 
the  parts  list  by  the  AWP  Impact  Index  or  any  of  its  variables.  However,  in  this  option, 
Cognos  would  not  have  to  query  the  D035K  system  for  any  infonnation  because  it  would 
be  contained  in  the  Inventory  Tracking  System,  as  depicted  in  Figure  16. 


Figure  16.  Option  #2  Method  of  Retrieving  Data 
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5.2.3  Which  Option  is  Better? 

Each  option  has  its  limitations.  First  of  all,  both  options  rely  on  human  input.  As 
a  result,  the  reliability  of  the  information  is  not  perfect.  Some  common  errors  known  to 
occur  are:  entering  the  wrong  delay  code,  not  recording  the  delay  at  all,  or  entering  the 
delay  at  the  wrong  time.  The  most  common  error,  based  on  discussions  with  technicians, 
work  leaders,  and  schedulers  is  entering  the  wrong  delay  code.  Technicians  use  certain 
codes  more  commonly  than  others,  so  they  will  sometimes  enter  a  code  that  they  use 
quite  often  instead  of  detennining  the  correct  code  to  select.  This  is  a  significant  threat  to 
reliability,  but  can  be  mitigated  by  training  and  supervision.  In  addition,  most  of  these 
errors  will  underreport  AWP  delays.  There  are  no  indications  that  this  error  would  occur 
more  frequently  on  certain  end  items,  which  means  that  the  error  should  be  independent 
of  the  end  item.  This  error  would  result  in  lower  AWP  delays  on  all  parts,  but  would 
probably  not  have  a  major  impact  on  the  rank-order  of  the  parts  list.  Of  course,  until  real 
data  is  collected,  this  argument  cannot  be  tested. 

A  limitation  unique  to  option  #1  is  the  fact  that  the  delay  time  recorded  would  not 
be  the  total  delay  time.  The  exact  moment  the  end  item  comes  to  a  work  stoppage  is 
recorded  with  the  R06  delay  code.  However,  recording  when  the  piece  part  is  no  longer 
backordered  does  not  provide  the  exact  time  the  work  stoppage  ended.  In  actuality,  the 
work  stoppage  is  ended  when  the  technician  can  begin  work  on  the  end  item  again. 

When  a  part  is  no  longer  listed  as  backordered,  it  simply  means  that  a  part  has  been 
located  to  satisfy  the  order.  The  part  could  be  in  transit  for  several  days.  This  reduces 
the  reliability  of  the  data.  By  definition,  this  underreports  the  AWP  delay  time.  In 
addition,  the  transit  time  for  certain  parts  could  be  consistently  longer  or  shorter  than 
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other  parts’  transit  times  depending  on  the  location  of  the  supplier.  As  a  result,  it  could 
affect  the  distribution  of  the  data  and  the  rank-order  of  the  parts  list.  However,  the  transit 
time  per  part  is  typically  much  shorter  than  the  overall  delay  time  on  many  AWP  parts.  It 
is  not  uncommon  to  wait  six  months,  twelve  months,  or  even  eighteen  months  for  some 
parts.  In  addition,  the  average  transit  time  (from  supplier  to  depot)  across  all  piece  parts 
could  be  added  to  each  delay,  thus  eliminating  any  biased  error.  Thousands  of  parts  are 
ordered  every  day  for  the  Oklahoma  City  Air  Logistics  Complex,  so  the  law  of  large 
numbers  works  in  our  favor.  In  short,  the  impact  of  not  including  the  transit  time  from 
the  supplier  to  the  depot  is  deemed  to  be  minimal,  especially  if  the  average  transit  time  is 
added  to  each  delay. 

A  limitation  unique  to  option  #2  is  that  it  relies  on  human  input  for  the  start  and 
stop  time  and  the  parts  causing  the  delay.  If  the  information  is  recorded  correctly  every 
time,  option  #2  is  much  more  accurate  than  option  #1.  Option  #2  allows  for  human  error 
on  four  pieces  of  infonnation,  whereas,  option  #1  only  allows  for  human  error  on  one 
piece  of  information. 

Reliability  and  validity  of  the  measurement  aside,  there  are  practical  limitations  to 
be  considered.  Option  #1  is  estimated  to  take  six  months  or  less  to  implement.  Its  cost  is 
not  yet  known,  but  the  Cognos  programmers  were  able  to  confidently  state  that  it  would 
be  significantly  cheaper  than  option  #2.  In  addition,  option  #2  is  estimated  to  take  four  to 
six  years  to  implement.  The  applicable  infonnation  systems  offices  of  primary 
responsibility  (OPRs)  and  programmers  at  Tinker  AFB  provided  the  time  and  cost 
estimates. 
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Due  to  the  high  cost  and  long  lead  time  of  option  #2,  the  author  recommends  that 
option  #1  be  implemented,  unless  the  increased  accuracy  of  option  #2  is  deemed 
necessary  by  the  users.  Option  #1  provides  relatively  accurate  infonnation  and  it  can  be 
made  available  in  a  much  shorter  period  of  time  at  a  much  lower  cost.  However,  the 
average  transit  time  should  be  added  to  each  delay  to  increase  the  accuracy  of  Option  #1 
and  to  avoid  any  biased  error.  Especially  given  the  tight  fiscal  situation,  option  #1  seems 
the  prudent  choice.  The  data  is  aggregated,  so  the  error  that  is  inherent  in  option  #1 
should  not  have  a  major  negative  impact  on  decisions  made  from  it.  The  main  argument 
against  option  #2  is  the  enormous  cost  of  just  gaining  the  capability  of  collecting  the 
necessary  data,  without  knowing  exactly  what  benefit  will  be  gained  from  it.  Hand 
argues  that  “. .  .if  the  aim  of  measurement  is  to  lead  to  an  improvement  of  some  process, 
one  must  be  certain  that  the  cost  of  taking  that  measurement  does  not  outweigh  the  gains 
to  be  made  through  knowing  it”  (Hand,  2004:  23).  He  goes  on  to  emphasize  that  this  is 
particularly  applicable  in  “managerial  and  governmental  contexts”  (Hand,  2004,  23).  In 
short,  if  option  #2  is  selected  for  implementation,  it  would  best  to  know  precisely  what 
benefit  can  be  gained  from  the  increased  accuracy  it  offers,  so  a  proper  cost-benefit 
analysis  can  be  conducted. 

5.3  Prioritization  Recommendations 

Using  only  one  variable  to  prioritize  piece  parts  ignores  the  other  important 
variables.  Therefore,  the  goal  was  to  combine  the  important  variables  in  such  a  way  that 
a  significantly  different  prioritization  is  created  that  considers  all  important  variables. 

The  results  show  that,  overall,  the  AWP  Impact  Index  prioritization  of  piece  parts  is 
significantly  different,  except  when  compared  to  the  AWP  time  prioritization.  The 
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correlation  between  the  index  and  AWP  time  was  not  extremely  high  (rs  =  0.65).  In 
addition,  this  could  be  overcome  by  adjusting  the  weights  assigned  to  the  variables. 
Regardless  of  the  weights  chosen,  subject  matter  experts  should  examine  the 
prioritizations  carefully  to  determine  their  correctness. 

Another  potential  limitation  of  an  index  equation  defined  as  a  linear,  additive 
relationship  of  its  constituent  variables  is  that  such  equations  can  give  inconsistent  results 
for  other  measures  when  they  are  in-turn  defined  as  products  or  quotients  of  the  original 
index  equation  variables.  Table  8  illustrates  this.  Note  that  each  of  the  weights  and 
variable  values  are  nonnalized  before  computation.  Suppose  that  an  AWP  impact  index 
equation  is  defined  as  the  weighted  sum  of  Frequency  and  Average  Time,  and  further 
suppose  that  a  new  measure  Total  Time  is  introduced,  where  Total  Time  is  defined  as  the 
product  of  Frequency  and  Average  Time.  In  this  scenario  each  part  has  the  same  Total 
Time  value  and  thus  a  ranking  computed  only  from  the  Total  Time  measure  should  result 
in  a  tie  for  piece  parts  A,  B,  and  C.  However,  Table  8  shows  that  the  AWP  impact  index 
equation  ranks  each  piece  part  differently,  in  a  strict  priority  order. 

A  final  insight  on  variable  values  and  weights  can  also  be  gleaned.  In  this 
extreme-valued  scenario,  the  weight  assigned  to  Frequency  must  be  more  than  100  times 
larger  than  the  corresponding  Average  Time  weight  to  prioritize  the  parts  by  frequency  as 
A,  C,  and  then  B.  However,  decision  makers  may  not  know  that  they  must  sometimes 
first  establish  weights  differing  by  several  orders  of  magnitude,  before  attempting  to  use 
such  weighted  ranking  schemes.  A  better  strategy  for  forcing  an  additive  index  equation 
to  focus  on  only  one  or  two  of  its  constituent  variables,  would  be  to  use  zero-valued 
weights  for  the  variables  to  be  ignored,  if  computationally  feasible. 
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Table  8.  AWP  Impact  Index  Limitation  Scenario 


Weights 

0.5 

0.4 

Total  Time 

AWP  Impact  Index 

Piece  Part 

Frequency 

Average  Time 

A 

1000 

1 

1000 

0.556 

B 

1 

1000 

1000 

0.445 

C 

10 

100 

1000 

0.05 

The  primary  recommendation  is  to  further  develop  the  AWP  Impact  Index.  For 
example,  a  multiplicative  equation  could  be  attempted.  Also,  a  further  reduction  of  the 
variables  may  be  beneficial.  Frequency  and  quantity  are  highly  correlated  (rs  =  0.90), 
which  means  having  both  of  them  doesn’t  change  the  prioritizations  much.  Since 
frequency  had  the  highest  relative  importance,  quantity  would  be  the  logical  variable  to 
remove.  Regardless  of  the  modifications  made  to  the  index,  analysis  would  still  have  to 
be  conducted  to  determine  if  the  prioritizations  were  significantly  different.  In  addition, 
subject  matter  experts  would  need  to  examine  the  prioritizations  to  determine  if  they  were 
correct. 

If  the  index  is  used  in  its  current  form,  the  weights  should  be  adjusted  and  the 
resulting  prioritizations  should  be  examined  carefully.  In  addition,  the  index  should  only 
be  used  to  narrow  the  list  of  piece  parts.  Subject  matter  experts  should  then  limit  the  list 
further  using  their  best  judgment  and  intuition.  The  index  should  only  be  used  to 
augment  knowledge,  experience,  and  intuition. 

All  four  methods  used  to  test  the  AWP  Impact  Index,  tested  for  differences  and 
commonalities.  None  of  the  methods  indicate  which  measurement  is  best  suited  for 
prioritizing  piece  parts.  Only  the  users  of  these  measurements  can  determine  which 
measurement  gives  the  best  prioritization.  While  the  AWP  hnpact  Index  may  not 
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produce  the  best  prioritization,  it  should  not  be  interpreted  to  mean  only  frequency  should 
be  used  to  prioritize  piece  parts  since  it  is  the  highest  priority.  To  do  so  would  inherently 
ignore  all  other  factors  that  drive  the  importance  of  a  piece  part. 

5.4  Future  Research  Opportunities 

This  research  only  scratches  the  surface  of  the  AWP  problem.  This  research 
provides  the  capability  to  identify,  track,  and  prioritize  piece  parts  causing  delays  in  the 
76th  Commodities  Maintenance  Group’s  repair  process.  The  next  logical  step  is  to  use 
this  capability  to  conduct  a  root  cause  analysis  of  the  piece  parts  with  the  highest  impact. 
Such  a  study  could  identify  major  opportunities  for  cost  savings,  process  improvement, 
and  increase  general  awareness  of  the  sources  of  AWP  delays.  In  addition,  this  thesis 
only  examined  the  76th  Commodities  Maintenance  Group.  As  a  result,  research  could  be 
done  to  detennine  the  how  or  if  AWP  delays  can  identified,  tracked,  and  prioritized  in  the 
other  units  at  the  Oklahoma  City  Air  Logistics  Complex  and  the  other  two  Air  Logistics 
Complexes. 

In  addition  to  expanding  the  scope  and  application  of  this  research,  the 
prioritization  methods  used  could  be  further  researched  and  developed.  This  research  did 
not  determine  if  the  prioritization  methods  used  produce  correct  prioritizations.  A  Delphi 
Study  could  be  a  useful  tool  for  such  research.  Also,  there  are  a  number  of  inherent  flaws 
with  each  prioritization  method,  so  there  is  plenty  of  room  for  improvement.  Such 
improvements  could  include  determining  a  better  way  to  combine  the  variables  identified 
in  this  research,  determining  if  different  variables  should  be  used,  and  determining  how 
sensitive  the  measurements  are  to  change. 
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While  conducting  this  research,  another  AWP  problem  was  identified.  This  thesis 
focused  on  AWP  delays  once  end  items  are  inducted  into  the  ALC,  but  delays  occur  prior 
to  their  induction  into  the  ALC  as  well.  This  was  briefly  mentioned  in  section  2.2.  This 
problem  was  identified  by  Mr.  Steve  Roberts  at  Tinker  AFB  and  he  has  made  some 
progress  towards  addressing  the  problem.  However,  there  remains  much  room  for 
research  and  improvement  (Roberts,  2013).  In  addition,  if  the  data  regarding  piece  parts 
causing  end  items  not  to  be  inducted  could  be  combined  with  the  data  on  piece  parts 
causing  delays  inside  the  Air  Logistics  Complexes,  it  would  provide  a  much  more  holistic 
view  of  the  problem.  This  would  greatly  enhance  the  Air  Force’s  awareness  of  the  root 
causes  behind  its  AWP  delays.  Such  awareness  is  the  necessary  first  step  to  solving  those 
problems. 

Questions  about  the  effectiveness  of  the  Kitting  step  in  the  repair  process  were 
raised  in  section  4.3.1.  Based  on  a  limited  view  of  the  Kitting  step,  it  appears  to  only 
shift  the  queues  from  the  Repair  and  Assembly  step  to  the  Kitting  step.  In  addition, 
increased  setup  costs  or  time  were  not  identified.  As  such,  no  direct  benefit  of  the  Kitting 
step  could  be  seen.  This  received  minimal  attention  in  this  thesis  because  it  was  not 
pertinent.  However,  research  could  be  done  on  the  effectiveness  of  the  Kitting  step  in  the 
76th  Commodities  Maintenance  Group  as  compared  to  other  units.  It  is  possible  that  the 
Kitting  concept  is  not  universally  applicable  to  all  types  of  shops  or  processes.  Such 
research  could  either  reinforce  procedures  currently  in  place  or  identify  weaknesses  in  the 
current  procedures.  Either  result  would  be  of  great  benefit  to  the  Air  Logistics 
Complexes  and  contribute  to  the  body  of  research  on  High-Velocity  Maintenance. 
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5.5  Conclusion 


When  this  research  began,  the  goal  was  to  compile  a  list  of  piece  parts  causing 
delays  in  the  Air  Force’s  Air  Logistics  Complexes  along  with  other  pertinent  information 
regarding  the  delays,  so  that  an  in-depth  analysis  of  the  underlying  causes  of  those  delays 
could  be  ascertained  and  addressed.  It  became  apparent  that  such  a  lofty  goal  could  not 
be  accomplished  within  the  time  constraints  of  this  thesis.  In  addition,  it  became 
apparent  that  even  the  most  basic  infonnation  about  AWP  delays  was  difficult  to  gather 
on  a  large  scale.  The  realization  that  followed  was  that  the  Air  Force  had  been  looking  at 
the  problem  from  only  one  perspective.  It  had  been  examining  the  AWP  problem  from 
the  end  item  perspective.  As  a  result,  all  of  the  information  systems  allowed  for 
summations  by  end  item,  but  not  summations  by  piece  part.  To  be  sure,  one  can  hone  in 
on  one  end  item  and  determine  which  piece  parts  it  is  consistently  or  currently  waiting 
on.  However,  no  one  interviewed  throughout  this  research  knew  of  a  way  to  turn  the 
tables  and  find  out  which  piece  parts  are  causing  delays.  The  Air  Force’s  infonnation 
systems  were  designed  to  track  data  by  end  item,  not  by  piece  part. 

This  thesis  identified  two  methods  that  will  enable  the  Air  Force  to  identify,  track, 
and  prioritize  piece  parts.  If  implemented,  this  will  allow  the  Air  Force  to  take  a  much 
more  holistic  view  of  the  AWP  problem.  Instead  of  tackling  the  AWP  problem  one  end 
item  at  a  time,  it  will  be  able  to  address  systemic  problems.  It  will  bring  a  level  of 
awareness  to  the  problem  that  has  not  yet  been  achievable.  In  addition,  one  of  the 
methods  identified  is  a  fiscally  responsible  option,  which  makes  its  implementation  much 
more  feasible.  It  will  not  require  modifications  to  any  infonnation  systems  and  it  will 
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only  require  a  minor  change  to  the  way  infonnation  is  currently  being  recorded  in  the 
76th  Commodities  Maintenance  Group. 

While  this  study  did  not  produce  an  aggregate  measure  of  AWP  impact  that  is 
ready  for  use,  it  did  initiate  the  conversation  and  thought  regarding  the  subject.  The 
measures  created  in  this  research  are  a  first  step  to  formalizing  how  AWP  delays 
negatively  affect  depot  operations.  An  enhanced  understanding  of  these  effects  should 
prove  useful  to  depot  leadership  in  day-to-day  decision  making  and  in  communicating  the 
importance  of  minimizing  AWP  delays. 
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